A tale of two GUIs.
The argument started with the web developers assertion that unit tests could test just about everything. And to look at her code, you'd almost feel the same way. As a TDD proponent, she had nearly twice as many tests as she has methods and classes. Not a single like was untested... Well, that probably isn't 100% true, but it damn near looked like it.
What she did not have, however, was a bug (of any sort) that lived longer than ten minutes, and even those were rare beasts. Sure, there were bugs in the overall system, where her code touched others in the organization, but not in hers. Every so often, entropy would take hold, and a requirement would be misunderstood, or a test that wasn't testing properly because of a misplaced semicolon or negation, and yes a bug would show it's ugly head. But it would be squashed with a new test that legislated it out of existence, never to return. So when you look at her development process, you could see that her testing advocacy was not unfounded jibber jabber... It was as solid as a rock, a fortress of evidence at which all arguments to the contrary would simply fail. But most importantly, it was a thorn in the side of the Swing developer.
He was a very good programmer. He had to be, because he was not just a swing developer, but a cross platform swing developer. That meant he not only had to deal with the nightmare of caveats, disclaimers and javadoc half-truths that is Swing on one platform, he had to find a way to deal with that on three platforms! So bugs were a common experience for him. They could be reduced in scope and danger, but they could not be eliminated. Sure he could write a unit test here and there, for the code that calculated the size of a box, or that that checked the web service for the login information... but when it came to the GUI itself, tests were simply impossible.
Impossible!? She would yell at him, how could it be impossible? It's all just inputs and outputs, right? But is it as simple as that? Logically it must be, because that is all a computer is... yet try as he might, he steadfastly refused to believe Swing was testable. Their argument was going on for days, and so I decided to look into the problem and see what I could see...
From my point of view, a swing GUI and a web GUI were essentially, from a fundamental level, really the same thing kind of thing. They displayed widgets that you could click on and type on and then things happen and the UI changes in some way. Different implementation, but same general purpose. So why were web-based GUIs so simple to test (with HTMLUnit, HTTPUnit, etc), but Swing-based GUI's so impossible?
Read on into the extended section to see why I discovered that, yes, Swing code is fairly untestable, almost by design... but that doesn't mean it's an inherant flaw in GUIs... there may just be hope after all in the next generation of GUI toolkits.Fundamental Differences
Well, if you notice the two tools that I mentioned above in terms of test frameworks for the web, you'll see the problem right away, well, the FIRST problem... Those frameworks don't test the rendering of the web GUI, they just test the layout definition or lower level interface protocol (respectively) that is used to create/talk to the GUI. (It is simply assumed that the rendering will not fail at the widget level. Of course, they will want to use div version A for windows and div version B for mac, and they can test that is working, but they never test the actual rendering itself... That is the browser's problem.) For Swing, there is no data transport layer, beyond the interal magic of calling a method, between the drawing of the UI and the underlying business logic of the widgets themselves. Oh sure, there is an attempt at an MVC architecture with many of the JComponents, which makes components a little less messy, but it's not the same clean, hard presentation/computation break that a web GUI gets.
Can Swing components be split into pure UI, pure business logic and a decoupled interface in between?
So, I thought, how hard would it be to teach Swing to fake it? Take a few of the components and split them into a layout definition, a UI renderer and a logic engine, all separated by something like a nice RMI interface. Basically I tried a quick test to see if I could decouple the abstract