Skip to main content

Multithreaded toolkits: A failed dream?

Posted by kgh on October 19, 2004 at 8:43 AM PDT

The question came up recently of "should we make Swing truly
multithreaded?" My personal answer would be "no", and
here's why...

The Failed Dream

There are certain ideas in Computer Science that I think of
as the "Failed Dreams" (borrowing a term from Vernor Vinge).
The Failed Dreams seem
like obvious good ideas. So they get periodically reinvented,
and people put a lot of time and thought into them. They
typically work well on a research scale and they have the
intriguing attribute of almost working on a production
scale. Except you can never quite get all the kinks ironed
out...

For me, multithreaded GUI toolkits seem to be one of the Failed
Dreams. It seems like the obvious right thing to do in a
multithreaded environment. Any random thread should be able
to update the GUI state of buttons, text fields, etc, etc.
Damned straight. It's just a matter of having a few locks,
what can be so hard? OK, there are some bugs, but we can
fix them, right? Unfortunately it turns out not to be
so simple...

From observation, there seems to be an amazing tendency
towards deadlocks and race conditions in multithreaded GUIs.
I first heard about this issue
anecdotally from people who had worked with the Cedar
GUI libraries at Xerox PARC in the early 80's.
That was a community of extremely smart people who really
understood threading, so the assertion that they were
having regular deadlock issues within GUI code was intriguing.
But maybe that was flawed data or an exceptional situation.

Unfortunately that general pattern has been repeated
regularly down the years. People often start off trying for
multithreading and then slowly move to an event
queue model. "It's best to let the event thread do the
GUI work."

We went through this with AWT. AWT was initially exposed as
a normal multi-threaded Java library. But as the Java team
looked at the experience with AWT and with the deadlocks and
races that people had encountered, we began to realize that
we were making a promise we couldn't keep.

This analysis culminated in one of the design reviews for
Swing in 1997, when we reviewed the state of play in AWT,
and the overall industry experience, and we accepted the
Swing team's recommendation that Swing should support only
very limited multi-threading. With a few narrow exceptions
all GUI toolkit work should occur on the event processing
thread. Random threads should not try to directly manipulate
the GUI state.

Why is this so hard?

John Ousterhout gave a great
Usenix talk on
Events versus Threads
in 1995 that explores some of
the tradeoffs between thread-driven and event-driven programming
and he correctly points out many reasons why
multi-threaded programming is hard and why event driven
programming can be simpler. I don't necessarily agree with
his analysis for all kinds of programs, but I do agree for
GUI programs.

The particular threading problems of GUI toolkits
seem to me to arise from the combination of input
event processing
and abstraction.

The problem of input event processing is that it tends
to run in the opposite direction to most GUI activity.
In general, GUI operations start at the top of a stack
of library abstractions and go "down". I am operating on
an abstract idea in my application that is expressed by
some GUI objects, so I start off in my application and
call into high-level GUI abstractions, that call into
lower level GUI abstractions, that call into the ugly
guts of the toolkit, and thence into the OS. In contrast,
input events start of at the OS layer and are progressively
dispatched "up" the abstraction layers, until they arrive
in my application code.

Now, since we are using abstractions, we will naturally
be doing locking separately within each abstraction.
And unfortunately we have the classic lock ordering
nightmare: we have two different kinds of activities
going on that want to acquire locks in opposite
orders. So deadlock is almost inevitable.

This problem will initially surface as a series of
specific threading bugs. And people's first reaction is
to try to adjust the locking behavior to resolve the
specific bugs. Let's release that lock there and
then lets use more clever locking over here. Well,
that is kind of a fun activity, but it is trying to
fight back an oceanic tidal force. The cleverer locking
typically turns into a combination of subtle races
(due to lack of locking) or clever and intricate
deadlocks (due to the clever and intricate locking).
We went through a bunch of that in 95-97.

Notice that the problems extends beyond the GUI toolkit
layers and also appears between the toolkit layer and
the application level. With great difficult one might
try to adopt a single lock for all activity within the
GUI layer, but the same problem then resurfaces a level
up.

So what's the answer? Well, at some point you have to
step back and observe that there is a fundamental conflict
here between a thread wanting to go "up" and other threads
wanting to go "down", and while you can fix individual
point bugs, you can't fix the overall situation.

This lead to the solution that the Swing team adopted
and which is used by most leading GUI toolkits:
run all GUI activity on a single event thread. This
means that in some sense all GUI activity becomes event
driven, and the "down" threads become just a new kind
of event.

This demonstrably works. It is possible to write complex
GUI apps that work reliably. Hurrah! But it does make
managing long running activities tougher. I wrote a
smallish Swing program that I use periodically to
selectively zap large boring attachments from my email archives.
I don't want to hang the GUI while it reads tens of megabytes
of emails, and I also want to display a progress monitor,
so I ended up having to carefully balance handing off big
activities to worker threads and handing GUI activities
back to the event thread. It is probably more complicated
than it would be if I had a magic multi-threaded library, but
it has the significant saving grace that it actually seems
to work reliably.

Subtleties

Are things really so black and white? Surely there
have been people who have used multi-threaded toolkits
successfully? Yes, but I think this demonstrates one
of the characteristics of the Failed Dreams.

I believe you can program successfully with multi-threaded
GUI toolkits if the toolkit is very carefully designed; if
the toolkit exposes its locking methodology in gory detail;
if you are very smart, very careful, and have a global
understanding of the whole structure of the toolkit.
If you get one of these things slightly wrong,
things will mostly work, but you will get occasional hangs
(due to deadlocks) or glitches (due to races). This
multithreaded approach works best for people who have been
intimately involved in the design of the toolkit.

Unfortunately I don't think this set of characteristics
scale to widespread commercial use. What you tend to
end up with is normal smart programmers building apps
that don't quite work reliably for reasons that are
not at all obvious. So the authors get very
disgruntled and frustrated and use bad words on the
poor innocent toolkit. (Like me when I first started
using AWT. Sorry!)

Another wrinkle: it is possible to have multiple simultaneous GUI
activities within a Java VM by using multiple event threads.
That works provided the different activities are almost
entirely isolated, have their own distinct GUIs (no shared
components or mixed hierarchies)
and provided the very lowest toolkit level can correctly
dispatch events to the right event thread with minimal
locking. This is useful in (for example) running
multiple applets within one JVM. But it isn't a very
general solution - most applications need to live within
the constraint of only a single event thread.

In this note I've most been covering why Swing and other
toolkits are essentially single-threaded. Chet recently
blogged on some related topics around

why multi-threading complicates user programs

and normally won't help raw graphics performance.

Also, before I forget, some people are probably remembering that
"processes and monitors are duals". Well, yes, it's true.
In some sense we are using the event thread to implement
a global lock. We could invert things, and create a global
lock that is equivalent to the event queue. This would be
fairly ugly and would require wide coordination and undermine
a lot of abstractions. But the larger problem
is that Java developers tend to use
multiple locks and if they are to preserve the equivalence
with an event queue model, they will need to follow various
non-obvious rules about how they interact with these
other locks.
The event queue model makes the central single lock much
more visible and explicit, and on the whole that seems
to help people to more reliably follow the model and thus
construct GUI programs that work reliably.

Conclusion

I guess the bottom line is that like many others I would
really like to see a flexible, powerful, truly multi-threaded
GUI toolkit. But I don't know how to get there - at this point
there is fairly strong experience that the obvious approaches
for multi-threading don't work. Maybe in future years people will
come up with a radically new and better approach, but for
now the answer seems to be that events are our friends.

            
            
            
            
            
            
            
- Graham

Related Topics >>