Global Transactions - X/Open XA - Resource Managers

원본출처 : http://www.aurorainfo.com/wp3/

Introduction

This document discusses Global Transactions. It describes the X/Open XA Model and provides some best practices that application designers and developers should be familiar with.

Return to Top of Page

Other References

Here are two extremely useful books that must be read by all n-tier application designers and architects.

Principles of Transaction Processing 350 pages

Authors: Bernstein, Philip A./ Newcomer, Eric

Publisher: Morgan Kaufmann Pub

ISBN: 1558604154

Published: December 1996

Transaction Processing: Concepts and Techniques 1070 pages

Authors: Gray, Jim/ Reuter, Andreas

Publisher: Morgan Kaufmann Pub

ISBN: 1-55860-190-2

Published: 1993

Transaction Standards

A transaction is used to define a logical unit of work which either wholly succeeds or has no effect whatsoever. It allows work being performed in many different processes, at possibly different sites, to be treated as a single unit of work.

Because the work being performed (within the bounds of a transaction) can occur on many different platforms and involve many different databases from different vendors, a standard has been developed to allow a manager process to coordinate and control the behavior of the databases. X/Open is a standards body that developed the Distributed Transaction Processing Model and the XA interface to solve the heterogeneous problem.

X/Open applications run in a distributed transaction processing (DTP) environment. In an abstract model, an X/Open application calls on resource managers (RMs) to provide a variety of services. For example, a database resource manager provides access to data in a database. Resource managers interact with a transaction manager (TM), which controls all transactions for the application.

The X/Open DTP Model defines the communications between an application, a Transaction Manager, TM, and one or more Resource Managers, RM. A RM is nothing more than a thing that provides access to shared resources and that supports the X/Open XA interface. The most common RM is a database (i.e. Oracle, DB2), however a print spooler can be a RM if it were to support commitment control through the X/Open XA interface.

The X/Open XA interface is a specification that describes the protocol for transaction coordination, commitment, and recovery between a TM and one or more RMs.

X/Open DTP Model

There are three components to the X/Open Distributed Transaction Processing Model:

. An application program that specifies actions which constitute a transaction.

. Resource Managers (i.e. RDBMS and Tuxedo /Q) which provide access to shared resources.

. A Transaction Manager that assigns identifiers to transactions, monitors their progress, and takes responsibility for transaction completion and for failure recovery.

. This diagram illustrates the three components and the arrows indicate the directions in which control flows.

The application communicates with the Transaction Manager to delimit transactions (begin, commit or abort). The Transaction Manager provides the application with API calls to inform it of the start, end and disposition of transactions.

The X/Open application does not establish and maintain connections to a database. Instead, the TP Monitor and the XA interface that is supplied by the RM Vendor, handle database connections and disconnections transparently. So, normally an X/Open-compliant application does not execute RM CONNECT statements.

The application communicates with the Resource Manager directly to perform useful work. The Resource Manager provides the means of communication between itself and the Application. The most common method is Embedded Structured Query Language (SQL).

The Transaction Manager and the Resource Manager communicate using the XA interface. The XA interface is a specification that describes the protocol for transaction coordination, commitment, and recovery. The two-phase commit protocol (2PC) is part of the XA specification.

The Tuxedo system provides the components and utilities for creating a Transaction Manager. The Resource Manager vendor (Oracle, Informix, etc.) must provide an XA compliant library that is used along with the Tuxedo utilities and components to build a Transaction Manager program.

Two Phase Commit Protocol (2PC)

Transaction Managers and Resource Managers use the two-phase commit protocol with presumed rollback. That is, if something goes wrong the TM and RMs involved in the transaction will always attempt to rollback their portion of the transaction (work).

In phase one, the Transaction Manager asks all Resource Managers if they are ready and able to commit a transaction. If a Resource Manager responds negatively, it (the RM) will automatically rollback any work it performed on behalf of the transaction and discard any knowledge it had of the transaction.

In phase two, the Transaction Manager determines if there are any negative reply뭩, and if so, instructs all Resource Managers to rollback. If all replies are positive, it will instruct the Resource Managers to commit.

X/Open Optimizations

X/Open specifies two optimizations to the 2PC protocol:

read only

If the Resource manager has only performed read operations and has not made any changes to the state, it can assert that it is not a participant and disregard the transaction.

1 phase commit

If there is only one Resource Manager involved in the transaction, the Transaction Manager may skip the inquiry part of the 2PC and simply instruct the single Resource Manager to commit or rollback. If the Resource Manager is not able to commit, the Transaction Manager may not know the outcome and the state will be left inconsistent.

Tuxedo automatically attempts a 1 Phase Commit optimization if only a single Resource Manager is involved in the transaction. The Resource Manager is not asked whether or not it can commit, it is simply told to commit. This is an automatic optimization in the Transaction Manager and can not be controlled by the user. There are however other optimizations that the user can control, when a two phase commit is used. See the tpscmt() and tpcommit() manual pages paying attention to the TP_CMT_LOGGED and TP_CMT_COMPLETE flags.

Establishing a RM Connection

In a Tuxedo Server process, the connection to the RM is accomplished by calling the tpopen() function. This function must only be called once at server initialization , and never within service functions. To close a RM connection, the tpclose() function must be called before the server process self terminates.

A Tuxedo Server process must never execute SQL statements such as CONNECT, COMMIT, ROLLBACK, SAVEPOINT, and SET TRANSACTION that affect the state of global transactions.

For example, the application must not execute the COMMIT statement because the TM handles commits. Also, the application must not execute SQL data definition statements such as CREATE, ALTER, and RENAME because they issue an implicit COMMIT.

Transaction Relationships

In the X/Open model, a TM can construct transaction trees by defining either tightly-coupled or loosely-coupled relationships with a RM.

Tightly Coupled

A \"tightly-coupled relationship\" is one where the same global transaction identifier, XID, is used by all processes participating in the same global transaction and accessing the same RM. This relationship maximizes data sharing between processes; XA-compliant RMs expect to share locks for resources used by processes having the same XID.

Tuxedo achieves the tightly coupled relationship via the server group concept; that is, work done by a server group on behalf of a given global transaction belongs to the same transaction branch; all the processes are given the same XID.

Loosely Coupled

In a \"loosely-coupled relationship\" the TM generates a transaction branch for each part of the work in support of the global transaction. The RM handles each transaction branch separately; there is no sharing of data or of locks between the transaction branches, hence deadlocks between transaction branches can occur. A deadlock results in the rollback of the global transaction. In Tuxedo, when different server groups participate in the same global transaction each server group defines a transaction branch; this results in a loosely-coupled relationship.

Global Transaction Identifiers (XID)

Tuxedo creates and manages global transaction identifiers (XID) that are then associated with local transactions within the RM.

A global transaction consists one or more units of work. Each unit of work (transaction branch) is identified by an XID. The XID consists of a global transaction identifier (GTRID) and a branch qualifier (BQUAL). This structure is based on the standard XA specification.

For example, the following is the structure for one possible XID of 1234:

[b]Component  Value  [/b]
          gtrid                       12  
          bqual                     34 
          gtrid+bqual=XID    1234  

Transaction Properties

A true transaction is said to have certain basic properties. ACID is an acronym for the basic transaction properties of Atomicity, Consistency, Isolation, and Durability as described below.

If a transaction violates any of these properties, undefined (usually bad) behavior may result.

Atomic

A transaction뭩 changes to a state are atomic: either all happen or none happen. These changes include database changes, messages and actions on transducers.

Consistent

A transaction is a correct transformation of the state. The actions, taken as a group, do not violate any of the integrity constraints associated with the state.

Isolated

Even though transactions execute concurrently, it appears to each transaction that others executed either before or after it.

Durable

Once a transaction completes successfully (commits), it changes to the state survive failures.

Application designers that are new to the distributed n-tier world usually break the Consistent and Isolated properties of a transaction. The way that a RM handles locks (page, record, optimistic/pessimistic, etc.) has a profound effect on the outcome of a transaction. Dirty reads, Phantom reads/writes and transaction time-outs are several symptoms of one or more ACID violations. It is way beyond the scope of this paper to describe the pitfalls of inappropriate application design. Again, I suggest you read the two books listed in the reference section of this document.

Transaction Types

There are three Transaction Types:

Flat Transactions

Chained Transactions

Nested Transactions

Return to Top of Page

Flat Transactions

The term \"flat transaction\" denotes the atomic unit of work performed by many entities and is controlled from one point. It contains any arbitrary number of actions that are taken as a whole and considered to be one action. It is considered flat because all worked is performed at the same level inside a single Begin/End Work bracket. The work may be performed by many different servers and may be distributed across many different platforms.

Tuxedo provides the following ATMI calls to control flat transactions:

tpbegin()

Called by the initiator to denote the start of a transaction. Once tpbegin() is called, communication with any other server can place that server in \"transaction mode\" (that is, the server\'s work becomes part of the transaction). Programs that join a transaction are called participants. A transaction always has one initiator and can have several participants. Only the initiator of a transaction can call tpcommit() or tpabort(). Participants can influence the outcome of a transaction by the return values they use when they call tpreturn(). Once in transaction mode, any service requests made to servers are processed on behalf of the transaction, unless the requester explicitly specifies otherwise via use of the TPNOTRAN flag.

tpabort()

Called by the initiator to denote the failure of a transaction. All changes made to the state of an object during the transaction are undone. This function may only be called by the initiator of a transaction. Participants express their desire to have a transaction aborted by calling tpreturn() with the TPFAIL flag. A protocol error will occur if a participant (not the initiator) calls tpabort().

tpcommit()

Called by the initiator to denote the success of a transaction. All changes made to the state of an object during the transaction are made permanent. This function may only be called by the initiator of a transaction. Participants can express their desire to have a transaction committed by calling tpreturn() with the TPSUCCESS flag. A protocol error will occur if a participant (not the initiator) calls tpcommit().

The transaction initiator may also suspend its work on the current transaction by issuing a call to tpsuspend(). Another process may take over the role of the initiator of a suspended transaction by issuing tpresume(). As a transaction initiator, a process must call one of tpsuspend(), tpcommit() or tpabort(). Thus, one process can start a transaction that another may finish.

Note:

The tpcancel() ATMI call does not have any affect on transactions. It is used to cancel a request which was sent via tpacall(). In fact, tpcancel() will fail if the caller is in transaction mode. You can not cancel a request which is part of the work of a transaction since that would implicitly cause the transaction to fail.

Chained Transactions

Chained transactions are similar to hard save points. The work is broken into pieces with each piece being under control of a flat transaction. Once a piece of work is complete it is committed or rolled back without regard to the state of the other pieces. Explicit Begin/Commit statements are not usually used. Instead the TP monitor begins a new flat transaction automatically whenever work is begun that is not currently under transactional control.

Tuxedo supports chained transactions by providing the TPNOTRAN flag to tpcall() and tpacall(). Use of the TPNOTRAN flag denotes that the requester does not wish the work performed due to the request to be under the same transactional control as the requester.

If the AUTOTRAN flag for the recipient of the request (that is, service routines) is not set to \"Y\", then the recipient of the request must make the determination whether or not it needs transactional control, in which case it may initiate a transaction using tpbegin(). If the AUTOTRAN flag is set to \"Y\" then Tuxedo will automatically initiate a transaction. Work performed by this service, and any work performed by services requested by this service will be under transactional control.

Tuxedo will automatically abort a transaction whenever a participant performs a tpreturn() with a TPFAIL flag.

If the chained transaction succeeds and is committed, and if the transaction under which the original requester is working is rolled back, the committed work remains durable. That is, the transactions are isolated and one has no affect on the other.

Chained transactions have merit in satisfying certain functional requirements. An example where chained transactions may be used is when there is a requirement to log information into a database that must remain durable regardless of the result of other work performed.

The following diagram shows a client that initiates a transaction and makes a request of Service A. Service A performs some work and makes a request of Service B to write some information that must remain durable if Transaction 1 fails. The structural dependencies of both transactions are not strong. Each behaves like a flat transaction.

It is conceivable that Transaction 2 will occur in parallel with Transaction 1 or at a later time. If the success or failure of Transaction 2 is going to have an effect on Transaction 1, then Transaction 2 should not be initiated, and all work should be under the control of Transaction 1.

Nested Transactions

Nested transactions are similar in concept to chained transactions with the exception that all sub-transactions are affected by the original transactions\' success or failure.

The following definition of nested transactions is taken from \"Nested Transactions: An Approach to Reliable Distributed Computing\" by J. Moss at M.I.T.

A nested transaction is a tree of transactions, the sub-trees of which are either nested or flat transactions.

Transactions at the leaf level are flat transactions. The distance from the root to the leaves can be different for different parts of the tree.

The transaction at the root of the tree is called the top-level transaction; the others are called sub-transactions. A transactions predecessor in the tree is called a parent; a sub-transaction at the next lower level is called a child.

A sub-transaction can either commit or roll back; its commit will not take effect though unless the parent transaction commits. By induction, therefore, any sub-transaction can finally commit only if the root transaction commits.

The rollback of a transaction anywhere in the tree causes all of its sub-transactions to roll back. This taken with the previous point, is the reason why sub-transactions have only A, C, and I, but not D properties.

The difference between chained transaction and nested transactions are primarily due to points 4 and 5 above.

Tuxedo does not currently support nested transactions.

Ensuring Data Integrity

Your application program must ensure the integrity of transactions that manipulate data. That is, the program must commit or roll back all SQL statements in the transactions. This might be impossible if the network fails or one of the systems crashes. For this reason Tuxedo and most Resource Managers implement the \"presume rollback\" paradigm. This is not a \'catch all\' however, and you must design your applications to consider dirty reads, phantom reads/writes, etc.

It is desirable to have as many as possible in-flight transactions occurring at once. The following objects help to ensure the Atomic and Durable ACID properties of each transaction:

Resource Manager and its transaction/redo log

Transaction Manager and its transaction log

XA and the 2 phase commit protocol

Because there may be many concurrent in-flight transactions, Isolation becomes a major concern. Consider the following:

Transaction T can be considered to be isolated if the following rules are followed:

T does not overwrite dirty data of other transactions. Dirty data is data that has not been committed yet.

Any data written by T is neither read nor written by other transactions until the write is committed. This will prevent lost updates.

T does not read dirty data from other transactions.

Other transactions do not overwrite data previously read by T before T commits. This is called an unrepeatable read. The application must provide safeguards for unrepeatable reads.

The use of a SELECT with the FOR UPDATE clause will lock the data and prevent other transactions from updating the data until transaction T has completed. Once T commits, all locks will be freed and another transaction may update the same piece of data, however transaction T no longer exists and therefore does not need the data anymore. Only when T commits will its view of the data be available for others to see.

The Conversational Communication paradigm and the Persistent Image using context data access method, described in the next section, along with the four rules described above, will ensure that the Consistent and Isolated ACID properties of transactions are safeguarded.

Data Access Paradigms

There are several items to consider when designing and implementing servers that read and update data from Resource Managers.

Clients and servers have different requirements for accessing large amounts of data. Most clients require the data a screen full at a time, while servers may require all of the data all at once.

There are three methods to handling large amounts of data:

Conversational mode (http://www.aurorainfo.com/wp3/23)

Advance through existing keys using context (http://www.aurorainfo.com/wp3/24)

Persistent Image using context (http://www.aurorainfo.com/wp3/25)

[colro=BLUE]Conversational Communication Paradigm

The conversational mode is not well suited for clients who need data one page at a time. Once a connection is established with a server, that server is unavailable for any other requests for the duration. It is possible for the client user to walk away from the workstation and leave the client connected for hours.

Conversational mode is better suited for batch clients whose data requirements are large and finite.

Consider a batch client that must read a file of 1000 rows and insert the data into the database. If a service were written to process the data and perform the required SQL, the client could connect to the service and initiate a half duplex conversation in which 50 rows at a time were sent to the service, until the entire file was processed. The client could then inform the service that it doesn뭪 have any more rows, at which point the service would initiate a graceful disconnection.

Advantage

Conversational mode can increase performance in cases where data must be returned from the service to another service or a batch client. A large amount of processing time is taken up opening SQL cursors, and in a conversational service context is maintained at the server level so the cursor only needs to be opened once.

Disadvantage

If the conversational communication method is used to select the update data, there is a danger of holding database locks for an unacceptable amount of time depending upon the application. Consideration to other running processes must be taken into account.

Advance Through Existing Keys

This method of data access is suited for clients who require a screen page of data per request. In this method, context is returned from the service and maintained in the client. The client and the service use normal synchronous communications. A fixed set of rows is returned along with a key that would allow the service to select the next set of rows on the next invocation.

The client is responsible for:

Saving the context key returned by the service.

Returning the context with the next request for more data.

If using FML, inform the service how many rows should be returned at one time. If using Views, the number of rows is fixed.

The service is responsible for:

If a context key is passed into the service, use it to open a cursor, select rows where greater than or equal to the context key, and pass the requested number of rows back.

If no context key is passed to the service, then open a cursor, select the first set of matching rows.

Pass back the rows, the number of rows returned in this invocation and the number of rows still remaining and the new context key.

Disadvantage

The major disadvantage of this approach is loss of Consistency and Isolation. Another transaction may interfere with the data that is still to be returned, or with the data that has already been returned. The ACID properties of the transaction are not enforced.

Some applications do not consider this situation to be very important.

Another disadvantage is the time required to continually re-open a cursor. Performance degradation increases with each open.

Persistent Image

This is a similar approach to the one above except Consistency and Isolation are not lost. Upon the first invocation, the service saves the entire data set to persistent storage, perhaps a flat file.

The context returned to the client contains the location of the data (file name) as well as the key. If a flat file is used, the key may be a simple row number. The client will return the context to any available service, which will retrieve the next subset of rows.

The client will inform the service when it no longer needs the data, and the service will remove the file.

Advantages

The advantages to this approach is that the cursor was opened only once, and the data can not be affected by another transaction. It safeguards against unrepeatable and dirty reads so all ACID properties are enforced.

Disadvantages

The major disadvantage to this approach is that the temporary storage may be left behind, and a mechanism to clean up derelict data must be implemented.

Another disadvantage is that on the initial invocation, extra time must be taken to write out the entire data set before returning the first subset.

Interfacing Tuxedo with a RM

Resource Manager vendors must publish certain information which is required to integrate with any TP monitor. Some of the information that is needed is:

Name of the XA switch

Name of the RM

Open Statement

Close Statement (optional)

Before an application may use a resource manager in a global transaction, a Transaction Manager (TM) specific to the RM must be built.

This section describes the steps required to build a Transaction Manager Server. Tuxedo provides a build utility (buildtms) which will enable you to build a transaction manager for any resource manager.

You simply provide the resource manager name and the resultant executable name, and all compiling and linking is performed automatically.

Before actually building a TM, there are certain configuration issues which must be addressed.

Tuxedo RM File

The RM file contains entries for all resource managers which may be accessed by Tuxedo applications. This file exists in the $TUXDIR/udataobj directory. Each entry consists of colon delimited fields and is terminated with a new line character.

The first field is a unique name of the Resource Manager. You may specify any name, however it should be understandable by other people in the future.

The second field is the xa_switch_t structure name.

The third field is a space separated list of libraries required to build the TM. These libraries are also used to build Tuxedo servers that access the RM.

The following is a sample from an actual RM file. Once the Resource Manager entry is properly entered into this file, a Transaction Manager may be built.

	# DB2 v5.2 on AIX 4.3
	DB2_XA:db2xa_switch:-L/sqllib/lib -ldb2
	# DB2 on AS400 
	DB2_XA:db2xa_switch:QTNXAAPI QTNXADTP QTNXAFNC

Building the Transaction Manager Server

A transaction manager server for the new resource manager must be built using buildtms and installed in $TUXDIR/bin.

The options to buildtms have the following meaning:

-v specifies that buildtms should work in verbose mode.

-o out_file out_file specifies the name of the file the output load module is to have.

-r rm_name rm_name specifies the resource manager associated with this TM. The value rm_name must appear in the resource manager table located in $TUXDIR/udataobj/RM. The entry associated with the rm_name value is used to include the correct libraries for the resource manager automatically and properly to set up the interface between the transaction manager and resource manager (using the xa_switch_t structure).

The following example will build a transaction manager for DB2:

buildtms -o $TUXDIR/bin/TMS_DB2 -r DB2_XA

The executable name of the newly built Transaction Manager Server is TMS_DB2. This is the name that must be used in the *GROUPS section of the Tuxedo UBBCONFIG file.

TMS Configuration Information

The Transaction Manager Server must be specified in the *GROUPS section of the UBBCONFIG configuration file.

The parameters that are required to start a TMS for a RM are:

TMSNAME

TMSCOUNT

OPENINFO

CLOSEINFO (optional)

The TMSNAME field contains the name of the executable that was built using the buildtms utility.

The TMSCOUNT field indicates the number of transaction managers that are booted to manage this group. The minimum number is 2.

The format of the OPENINFO string is dependent on the requirements of the vendor that provides the underlying resource manager. Tuxedo requires that the OPENINFO string required by the vendor must be prefixed with rm_name, which is the first field in the RM file for this interface (i.e. DB2_XA), followed immediately by a colon.

Example:

*GROUPS  

\"DB2_GRP\"                 LMID=\"aurora\" GRPNO=4 
                                OPENINFO=\"DB2_XA:RDBNAME=DEV  
                                USER=joeuser PASSWORD=joepwd 
                                SRVPGM=TUXLIB65B/LIBTUX\" 

                               TMSNAME=\"TMS_DB2\" 

                               TMSCOUNT=3 

Once the TMS server group has been defined, Tuxedo servers may be placed in SVRGRP. All servers placed in a TMS server group must have been built by supplying the -r option to BuildServer.

When the Tuxedo application is booted, the TMS processes will be started before any server that accesses the associated RM.

Identifying Server Groups

Clients may begin, commit and abort transactions, however for the purposes of this discussion regarding server groups, only servers will be shown.

Tuxedo requires servers and transaction managers to be grouped together (logically associated with each other) when global transactions are implemented. In the most simple case there will exist two transaction managers (TM) and one resource manager (RM), however in real world applications there are normally many different types of TM\'s and RM\'s that an application must access to complete its function.

Depending on the number of application servers there are that requires access to the RM, and the load of each request, more than two TMs may be booted at the same time for the same instance of the RM. Additional TM뭩 are considered slave processes to a single master TM process within the same group. The TM that is used when the requester first begins a transaction, is the coordinator for all other TM뭩 involved in the transaction.

The above figure shows the following:

Three applications. APP1 and APP3 consist of one server process each. APP2 consists of 4 server processes.

Multiple instances of TMS_ORA running within a single server group, all accessing the same RM.

Multiple instances of APP2 servers in server group 1, which initiate requests to APP2 servers in server group2.

The APP2 server in server group 2 is a participant of the transaction begun by an APP2 server in server group 1.

The TMS_ORA TM will coordinate the transaction with the TMS_QUE TM.

If APP1 were to call APP2, then the same XID would be used to access the Oracle database (tightly-coupled), and the same GTRID would be used but a new BranchID would be created to enqueue a message (loosely-coupled).