The Source for Java Technology Collaboration
User: Password:



Mark Little's Blog

Transactions are your friend

Posted by marklittle on March 02, 2006 at 01:25 PM | Comments (13)

After having been doing this for 20 years, I can say that transaction processing has got to be one of the most difficult middleware components to persuade developers to use. There are several reasons for this, but probably the most important is that, unlike something like caching or security, you don't see the benefits transactions bring until there's a failure. Unfortunately (or fortunately, depending on your perspective), failures don't happen that often, so actually demonstrating the utility of using a transaction processing system is made even more difficult. Furthermore, unlike something like security, you're unlikely to be refused access to a resource because you're not using it within the scope of a transaction.

However, thanks to the inefficiencies of natural selection (humanity is not perfect yet) and the beauty of entropy (all things decay), failures will always happen and so transactions will always be needed: all we can ever hope to do as technology advances, is reduce the probability of a failure occuring. Therefore, as a developer you've got to weigh up the likelihood of a failure (any failure) happening and corrupting your application versus the perceived cost (commercial and the overhead of restoring the system to good health) of using transactions. If you want to take the risk, then don't use transactions; but likewise, don't forget that they do exist to help you.

Now you may think that replication of resources/objects/servers could be used in place of transactions, but that isn't the case. Replication and transactions can be complimentary, but they're not a replacement for one another. Transactions guarantee consistency even in the presence of complete system failures, but you won't necessarily get forward progress. However, replication offers (though cannot guarantee) forward progress in the presence of a finite number of failures. So I would argue that if you are replicating updatable data, then you should definitely consider transactions as well.

Which leads us to another problem with selling the idea of transactions, that I've blogged on before: the notion that they slow your application down. Combined with the first problem I mentioned, I've often heard this referred to as "I get nothing for something" syndrome: you get the overhead of using transactions, but you just don't see the benefits they bring (which, looked at from some perspectives, is an entirely logical conclusion to make). Of course transactions slow down your application: I've
discussed this before
, but if you just think about what they have to do in order to guarantee consistency in the presence of failures, it makes sense: there really is no such thing as a free meal!

Transaction processing systems have been the backbone of significant areas of computing infrastructure for decades. A lot of these places (finance, telecos etc.) made the trade-off between performance and reliability because there was never a trade-off to be made: if they corrupt data (e.g., lose updates to stock trades), then institutions lose business. Now obviously that's not the case everywhere and there are applications where failures really don't matter (e.g., stateless interactions). But in general, you need to think about the effect of failures on your applications and although transactions are just one of the techniques you could use to help tolerate them, with the JTA they are a core component of J2EE. So rather than come up with ad hoc solutions, it may be better to try to leverage tried-and-tested techniques and associated implementations.

Following on from this is the oft heard statement: "everything I do is within a single VM, so I don't need transactions". This is definitely an education issue, where distributed transactions have become synonymous in the minds of many people with transactions. Most people can see that if they're accessing resources/participants across physically distinct machines or processes, there's a need for transactions to coordinate simultaneous updates to state. In a local (single VM) environment, the need is often overlooked. But it is still there: in many cases, even within the same VM, applications use and modify data from multiple different sources, and in that case, you need the benefits that transaction processing provides. Distribution just makes it more obvious that independent failures can cause problems. But they're still there in the local case; you may just have to look a little harder.

Plus, transactions get a lot of bad press for overheads that really don't exist in all cases. All commercial grade implementations support a number of significant optimisations to improve performance in the 80-20 case. For example, if there's only a single participant in a transaction, then the notorious two-phase commit protocol goes away and we run with a single phase. Then there's the read-only optimization: if a participant didn't modify any data, then it can drop out of the transaction "early". Plus, there are some implementations that have evolved over decades to offer many other performance features, such as lightweight coordinators, nested transactions and non-durable participants. The intention (mirrored by Microsoft's work with Indigo transactions) is to make transaction implementations so lightweight, with low overhead that they'll become a natural part of the infrastructure (and in the case of Microsoft, that'll mean in the operating system). We've already seen them moving into hardware, so this makes a lot of sense too.

As I've said before, think of transactions like an insurance policy: compared to how much time, money and effort you may lose by not using them, the cost of using them may be well worth it. Obviously there's a tiping point on any graph of cost of using transactions versus advantage they may bring, and that point is going to be very dependent on your application. But consider it nonetheless.


Bookmark blog post: del.icio.us del.icio.us Digg Digg DZone DZone Furl Furl Reddit Reddit
Comments
Comments are listed in date ascending order (oldest first) | Post Comment

  • Transactions are my friend too. Most certainly during application testing.

    Posted by: ttsgosadow on March 03, 2006 at 01:04 AM

  • So is this for things like maintaining test database consistency, or do you use them as a core component in your application itself?

    Posted by: marklittle on March 03, 2006 at 02:04 AM

  • I see many developers confuse transactions with locking, and as locking limits the scalability of the app (by limiting the number of concurrent readers and/or writers) transactions would limit scalability as well. :-(

    Developers also got used to ramdom failures anywhere, and as most failures get rarer as hardware improves, they tend do not perceive when failures are caused by lack of proper transation coordination.

    The same happens with race conditions in web apps. Ever tried to show a newbie developer why the use of <%! in JSP pages is dangerous?

    Posted by: flozano on March 03, 2006 at 07:32 AM

  • Hi Mark,

    What tends to be overlooked is the requirement to have a transaction to delimit (define) a business transaction that involves more than one database interaction - even read-only operations.

    I have seen too many development teams not using transactions for particular business transactions involving more than one database interaction because each interaction is read-only. Read-only operations such as SELECT may require a resource transaction if the two read-only operations have some data dependency (relationships). Imagine reading in a WorkOrder in one (implicit) transaction (connection.autocommit=true) and then reading the list of WorkItems in another (implicit) resource transaction with the application workflow code always expecting the workItems.size() > 0. This could be broken if between the first read and the second read another READ-WRITE transaction deletes both the WorkOrder and WorkItems.

    JXInsight is the first product on the market that can help test teams verify the database transaction semantics of a business transaction (use case) are correctly realized within the component execution a J*EE application.
    plug
    Regards,

    William Louth
    JXInsight Product Architect
    CTO, JInspired
    "J*EE tuning, testing and tracing with JXInsight"
    http://www.jinspired.com

    Posted by: williamlouth on March 03, 2006 at 08:19 AM

  • You're right to an extent about locking, but concurrency control of one form or another is something that many developers understand and see the benefits of. You don't use it and you can get into a lot of trouble, even without failures.

    Posted by: marklittle on March 03, 2006 at 08:27 AM

  • Hi William. Nice product plug! What you describe is certainly an issue, but it falls back to concurrency control: I'd have the same problems if I didn't use transactions at all, but I could solve it using a suitable concurrency control mechanism. If you don't use CC then you'll get burnt and developers will see that fairly easily.

    Posted by: marklittle on March 03, 2006 at 08:34 AM

  • Hi Mark,

    I hope the product plug did not spoil the posting for you. I did try to bracket the test with the plug tag but it was stripped in the translation.

    The two most important components of a transaction system is concurrency control and recovery. My posting was in relation to the first.

    The scenario I have described is not as rare as one would like or expect. It is compounded by the fact that this particular aspect of a busines use case is generally not clearly specified by the analyst or is (has been) extremely hard to verify and test. That is why our product treats traces/transactions as history (sequence) patterns that can appear again and again during the runtime execution of an application.

    William

    Posted by: williamlouth on March 03, 2006 at 11:11 AM

  • Hi William. No, the plug didn't spoil it.

    I'd disagree about the two most important components, simply because I'd assume that implicit in your "recovery" there has to be 2PC: I'm not really interested in recovery if it brings everything back up in a totally inconsistent/non-atomic state.

    I do agree that what you mention is a problem, but as I said, it's not strictly just a problem with transactions: it's a classic concurrency control issue along the lines of (though not identical to) the dining philosophers. So in that situation I'd have thought what you were plugging had wider applicability.

    But the central point of my original post was that the durability/recovery/2PC aspect of transactions is actually the hard part to sell, simply because in normal day-to-day executions, people don't see the necessity. If you take the CC out of their control, then with or without transactions developers will see a problem: just throw a few hundred concurrent threads at the system - a much more likely situation than a failure occuring. Like I said, I've been doing this for several decades and I've never once heard anyone complain about the necessity for locking (even two-phase locking): they may complain about how it is implemented - classic locks or timestamps, for example, but never about the fundamental need for it.

    Mark.

    Posted by: marklittle on March 04, 2006 at 01:05 AM

  • Hi Mark,

    But if the business transaction (read-write) is partitioned incorrectly across multiple (implicit) resource transactions then it is a problem both in terms of concurrency control and durability/recovery when we consider the overriding business transaction context. There are many "enterprise" products on the market that do this even though this is not expected by the customer. We have been involved in the profiling and transaction analysis of some large enterprise software packages where we have shown both the customer and the product architects of the 'real' transaction behavior of their systems in motion - 'chopping of transactions'. All involved understood the importance of concurrency control and durability/recovery but none had tested whether the design (expected behavior) matched the actual implementation (actual behavior). Again I am sure that most professional developers understand the concepts but understanding and verification are two different things. This is also further compounded by the simplicity in changing transaction demarcations for J2EE components and resource connectors.

    William

    Posted by: williamlouth on March 04, 2006 at 05:08 AM

  • By the way, I liked reading your Java Transaction Processing book though I have yet to finish reading it. Take no offence as that could be said for all the 'reference' books I buy. It is situated alongside other tx reference books: Transaction Processing (Jim Gray...), Transactional Information Systems, and Principles of Transaction Processing. - William

    Posted by: williamlouth on March 04, 2006 at 05:19 AM

  • Hi William. Yes, I agree that partitioning is important and often overlooked. You should take a look at some of the "enterprise" workflow systems out there - the patterns they encourage make me shudder ;-)

    Mark.

    Posted by: marklittle on March 04, 2006 at 05:46 AM

  • Perhaps the developers don't use transactions because transactions imply thinking of a big picture of what needs to be accomplished and most are focused on a part. This would elude to a need for transaction demarcation within an architectural design of the system.

    The developer would then be able to determine the whether they are developing within a transaction or not. Also I have seen a layered approach to transactions at some places with imply the transaction is queued from a source that is not so trusted and a followup transaction is schedule to proceed within an official commit or rollback of the queued transaction.

    This would imply that the transaction is started within one system and followed up with a 2PC on another system. Which seems to be addressed by some elements within the WS-TX.

    Posted by: dmarchant on July 20, 2006 at 12:50 PM

  • Perhaps the developers don't use transactions because transactions imply thinking of a big picture of what needs to be accomplished and most are focused on a part. This would elude to a need for transaction demarcation within an architectural design of the system. The developer would then be able to determine the whether they are developing within a transaction or not. Also I have seen a layered approach to transactions at some places with imply the transaction is queued from a source that is not so trusted and a followup transaction is schedule to proceed within an official commit or rollback of the queued transaction. This would imply that the transaction is started within one system and followed up with a 2PC on another system. Which seems to be addressed by some elements within the WS-TX.


    acrobat
    adaware
    antivirus
    antivirus
    ares
    avatar

    avatars
    clone
    divx
    download
    edonkey
    emoticone

    emoticones
    emule
    firefox
    gratuit
    gratuite
    icq

    kazaaa
    limewire
    logiciel
    logiciels
    messenger
    morpheus

    msn
    nero
    realplayer
    shareaza
    skype
    spybot

    telechargementnu
    telechargerdas
    pitelechargerant
    tytelechargerant
    frtelechargerant


    vlcolo
    winampaww
    rrewinmx

    tiwinrar
    powinzip

    dedamateur
    amateurdes
    eeranalu
    aseanime
    okoasian
    duiass
    babere6

    fubbw5
    bdsm55
    derbeach4
    bisexualpo
    bizarrecu
    axsblack


    deblonde5
    tyhlowjob
    cobondage
    gubooty
    vhuvukkake
    ddecartoon

    buexotic
    celebriteuy
    plitoris44
    crwcloseup
    edecoock
    deacouple

    qecreampie
    sripumshot
    judeepthroat
    ggildo66
    tredoublepenetra
    verdrunk
    lopebony

    treethnic
    bufaceh
    mifacial
    mhhofat
    frefemdom
    fetishhh55

    dreffmm
    tyfingering
    fist timefre
    istingde44f
    flashingsa
    footjobre

    uckingdas
    banggangtre
    gay23i
    gayssw
    gloryghole
    gugranny
    hairypo

    andjobde
    hardcorereh
    hentaityy
    housewifesaw
    indianerrh
    interracial223

    guujapanese
    latexree
    latinatyy
    lesbianvgh
    lingerieww

    guumaid
    ffmardigrasre
    pasturbation
    yumature
    remidgeth
    demilfg

    frmmf
    modeluy
    nudeerh
    nudistcre
    oralvre
    orgyoko

    greoutdoor
    pantiereh
    pissingdas
    plumperdes
    ornstaroko
    pregnantsas

    repuffy
    ushsy
    redheadca8
    rimjobse
    schoolgirlas44
    shavingde

    sshemale66
    showerbathh
    smokinghh
    softcoreh
    spankinghh
    squirting99

    eshstories
    tystrapon
    reswinger77
    mateenghy
    nnthong34
    uythreesome

    destits
    toesssa
    toonty44
    toynum
    trannyoi
    uglyplo44

    ervirgin
    tyuvoyeur
    drewebcam
    loweird9
    yuwetshirt

    pofilmgratuit
    piorngratuit
    poxxxgratuit
    paornogratuit
    losexgratuit
    tysexegratuit


    ggyxgratuit
    oitelechargement
    plotelechargemen
    jutelechargement
    atelechargemorno

    eetelechargement
    etelechargemsexe
    utelechargementx
    tytelechargermov
    lltelechargerfil

    aytelechargerorn
    ketelechargerorn
    vtelechargersex5
    otelechargersexe
    vytelechargerx
    telechargerxxx76


    gtdosex55
    pororno
    lamovie77


    sdfaged
    ffamazing22
    adanime
    ass3hole334
    hybabe00
    jubackseat

    juibang
    lobathing
    frbeachhh
    vebeauty
    bybeaver5
    hubed99

    bubiggtit
    lobimboh
    nubizarre
    beboyh
    bybusty
    yybutt

    nucheerleader
    michickk
    kichinese
    lochubby
    beclassyy
    nyplit

    frcoed
    buicorset
    kicum77
    likunt
    nucurly
    cadick77

    dedirtyy
    sedolll
    aedorm
    redressss
    ggdrilled
    trdrinking

    uyfoot
    ffgirlfriendd
    iglasses55
    gorgeous77ii
    goth22uu
    grannyf0

    hiddenvv9
    hirsute56
    hooterby66
    hornygr55
    hugegats
    jerkingjju

    jizas
    juicyce66
    kinkyffr
    kissing55
    leg66ii
    lickggh


    malehy
    nasturbatingr
    melonfu
    nistressre
    momase44y

    nakedaas
    naturalder
    naturetre
    naughtyvu
    nippleswe
    nymphoww

    oiled22
    older99uu
    orgasm32
    orientalvb66
    outsiderr66
    pantytr

    penetratefrr
    petitekki
    pinklo88
    plumppe
    ornstarve
    prettyty77

    privateru
    skinnyde
    szlutda44y
    smokesa55i
    spankserr
    zzs3permfrss

    spreade3
    stockingrr5
    stripe4ii
    strippertr
    studses
    studentad

    suckingase
    sybiancv
    tall334y
    teaseuy
    toplesszass
    trannycre

    tubash5h
    twinkse
    undressingre
    baginao
    bibratorry
    vixentre

    wwhooreedd
    wivesas


    Posted by: sallee on October 11, 2006 at 02:13 AM



Only logged in users may post comments. Login Here.


Powered by
Movable Type 3.01D
 Feed java.net RSS Feeds