John D. Mitchell's Blog

Extreme Programming Archives

Tests, Specifications, Typing, Oh my!

There's some interesting discussions taking place on the nature of tests. Brian Marick distinguishes between tests as specification vs. tests as examples. Michael Feathers asks if type systems in programming languages are really tests.

Kevin Lawrence (and the Agitar crew) talk about the philosophical contention between the notions of For All and There Exists. That is, the difference in mindset of existentialists vs. universalists. Looking at debates in e.g., the extreme programming world, there's a lot of confusion and arguments back and forth that stem from this constructivist vs. deconstructivist conflict.

The biggest tragedy in these debates, IMHO, is that people on both sides of the fence polarize and calcify in their self-righteous positions. The fact is that we need some amount of both approaches to succeed. For example, writing test-first leads to horrible code if you don't also refactor as you go. Accretive unit tests that aren't themselves refactored leads to big, ugly and unmaintainable test suites. More simply, positivist (aka "garden path") tests must be balanced with deconstructivst (test (to) destruction) tests.

Stepping up a level or three, it's even better if, rather than merely mitigating and ameliorating problems, we change the game such that it's hard/impossible to even articulate bad ways and trivial/easy to articulate the garden paths. That is one of the biggest benefits to taking a linguistic approach to development.

Humane interfaces, simplisticity, and domain languages

Elliotte Rusty Harold has touched off a small war in his response to Martin Fowler's recent entry on so-called Humane Interfaces.

One facet of the debate is an example comparing the equivalent "List" classes from Ruby (Array) and Java (java.util.List). Java's list class has 25 methods while Ruby's has 78 methods. Martin uses that fact to conclude that Ruby's list class is somehow more "humane" while Elliotte's thinks it's just bloated and that a minimal interface is better in terms of how people work.

Martin's primary argument for the "more is better" approach is that:

Humane interfaces do more work so that clients don't have to. In particular the human users of the API need things so that their common tasks are easy to do - both for reading and writing.

More buttons/methods does not make an object more powerful or more humane, quite the opposite in fact. Simplicity is a virtue. Smaller is better.

As you might have guessed, I think both of them are partially right and that there's something even more important that they are really bringing up that should be discussed.

For example, list.first() is actually more humane/usable/readable/etc. than list.get(0). Why? Because the intent needs to be clear and obvious to humans, not just the compiler. Even worse, crap like list.get(list.size() - 1) is just plain wretched compared to list.last() -- intent, clarity, complicatedness, easy to get wrong (off by one), etc. Also, look at how many parts of the list abstraction are "leaked" in just those two examples such as linear positioning, indexing, zero based indices, the first element is "always" at position zero, reliable sizing, etc.

However, Elliotte is completely right that having 78 methods in any class is an atrocity. Something that has that much surface area is way too complicated for humans to keep manageable. In addition, it also sets a bad example for coders learning the recommended ways of doing things -- i.e., "just throw anything you feel like in there."

Going to the opposite extreme of a bare minimum, necessary set of methods is also too simplistic. For example, Elliot throws downs an image of various remote controls, two fairly complicated "universal" remotes and a minimalistic one from Apple. But, who gets to chose what that minimal set will be for everybody? In software, almost everyone will end up wasting time and introduce bugs by writing their own versions of truly common bits of code. Software is, in this regard, much more of an engineering practice than a mathematical reduction.

Hopefully it's obvious that there's a reasonable middle ground. Like any good standard, deciding on what to put into the core library should be about codifying truly common behavior rather than what might sound good to any one special interest group. Other important tools in this effort are things like good design principles and refactoring. [I find it ironic that Elliot brought up the issue of refactoring in a debate with Martin.] Also, the both extremes miss out the addressing the specific needs of the context in which the code is being used: a pro using something everyday vs. a serious hobbyist vs. a random user vs. a half-blind grandmother with rheumatoid arthritis vs. .... Context matters.

Alas, arguing back and forth over those sorts of details makes it easy to miss a fundamental, crucial point: no software (library, application, language, operating system, or whatever) can be all things to all people. Fighting that war is not only pointless but is one of my definitions of insanity. The point of a chunk of good software is to enable the effective and efficient creation of more good software and to help inhibit the creation of bad software.

So, how do we then build up our own code to fix whatever shortcomings we find? Build our own libraries on top of whatever the core gives us which provide the clean abstractions and domain specific languages to get our jobs done, well.

Code Naked

Finding yourself naked, in public, is a dream/nightmare that many people have. It could just be me but when I was first starting to program, I had a nightmare that not only was I coding naked but my code was naked, too. Partly in response to that, I became much more diligent about writing excellent code so that I'd never be embarrassed by my software.

Over the years, I've joked with various people that I "code naked" but most folks stop at the (all too frightening) image of me coding while naked. Alas, I've never thought about a more palatable phrase enough to come up with anything worth mentioning. Various industries like to use the term transparency but methinks that's too opaque and wishy-washy.

A student of Bob Koss comes to the rescue with the term Refrigerator Code:

It's code that you’re so proud of that you want to take it home and hang it on the refrigerator, right alongside of your children’s drawings.

On the other hand, one of the underlying reasons of Why Software Sucks is the fact that most software written is really what I call Toilet Code:

It's code that's so mediocre that when somebody encounters it, they just want to flush it down the toilet.

DSLs feelin' groovy (or, graduating from elementary school)

Ben Galbraith has posted the first of a series of blog entries about How I Learned to Love Domain-Specific Languages. It's great that more and more people are starting to see the value of explicit, focused languages over ridiculously inhumane "formats" like XML. Hopefully, we're finally reaching a tipping point.

Explicit DSLs feel weird to a lot of programmers because there's been so little mainstream focus on them. I.e., as shown by one of the comments, developers have been herded and otherwise sucked in by shiny-looking tools (by poor education, management, laziness, peer-pressure, ignorance, lack of training, marketing hype, etc.) and haven't (consciously) realized the power of domain languages. It's amazingly odd to me how little energy has been applied to languages among mainstream developers given how much programmer time is spent arguing about the minutia of programming languages and tools.

The fact is that we're already surrounded by and are constantly implementing "DSLs". Look at the "language" of printf and friends, the declarative "specification" of makefiles, the myriad "protocols" that we deal with everyday like HTTP, SMTP, SSH, and FTP, the "APIs" of code libraries, the "design patterns" embodied in frameworks, the analogies and "metaphors" we use to described software architectures, the implicit languages that we create each time we define a class, the jargon we use to talk with each other, etc.

A big part of the problem that I see happening right now is that too much of the discussion around "DSLs" is being framed as some sort of "either/or" / "black/white" conflict when it's really just a more conscious and explicit approach to things that we've already been doing. Whether it's the hype juggernaut of Ruby on Rails or the Java is old, boring, bloated, etc. ideas exemplified by Beyond Java or the "IDE" wars between Eclipse, NetBeans, IntelliJ IDEA, and Emacs, or whatever, the biggest issue with this "us/them" thinking, IMHO, is that people are fighting the wrong fights. The leverage that matters most is the ability of developers to think and communicate clearly with themselves, each other, systems, business folks, and users. Biologically and sociologically, human are built to be linguistic.

That is, languages are fundamental to how we work internally and with each other. Sure, we have various tools to help us communicate but isn't it clear that e.g., PowerPoint isn't the point, it's just a tool — and, alas, a tool that usually induces poor communication rather than enriching conversations). On the other hand, look at the "modern" killer apps and how they are all about helping us (manage our) communicating: email, web, blogs, P2P, wifi, cell phones, faxes, VoIP, agile/XP, open source, etc. I.e., we've graduated from the elementary school building blocks (word processing, spreadsheets, databases, Belief of Control, etc.) to the middle school of communication. Now, we just need to learn and develop languages and tools built around this new level of understanding and put aside our old, comfortable, but ultimately dead-end habits.

Belief of Control

Well, I am a sucker for discussions about risk and software development. There are some interesting tidbits in Tiwana and Keil's: the one-minute risk assessment tool article for the ACM's Queue magazine. Alas, there are some fundamental problems with the article. So, definitely read it, but with a few grains of salt :-).

First off, while they noted the (potential of) self-selection bias, that's buried in a sidebar.

In conjunction, they completely gloss over the fact that they are only dealing with the stated assessments of the participants (i.e., the upper-level managers) of these projects. That is, they miss the entire point that the stated reasons rarely coincide with the real reasons.

In addition, they only talked to high-level managers — where's the talk with the people actually doing the work?

Building on those mistakes, of course the outcome of the research supports the notion that the managers can exert a lot more control over the success or failure of projects. Now, you're probably nodding your head and agreeing — fine, that fits our underlying intuition but the article and those managers are making rationalizations — not revealing reality. The first problem with the rationalization is the Belief of Control. I.e., the belief that managers (and other people in positions of authority) have that they can control everything. [Though, I love the ending of the article with the paraphrase of the Alcoholics Anonymous prayer. :-)] On another hand, in the vast majority of projects, of course the managers can control the success or failure of the project — because the managers are the cause of most of the problems. That is, the managers consciously and unconsciously set things up to fail through e.g., ignorance, neuroses, politics, greed, and just plain incompetence. Even worse, as the project unfolds over time, the tendency of managers is to apply those same broken approaches to "fixing" the problem and thereby usually just make things worse.

"We have met the enemy and he is us." —Walt Kelly, Pogo

Use less milk?

My little girl has this habit of pouring way too much milk into her bowl of cereal. Then, she whines and complains when we tell her to drink up the extra milk after the cereal is gone so it doesn't go to waste. Yesterday, she got quite snippy when I dared to suggest that she try pouring less milk into the bowl.

Gee, she sounds like a lot of managers and developers of software.

Though, to be more precise, in most software situations, it's not that they pour too much milk but, rather the converse: trying to eat way too much cereal for the given amount of milk. Of course, it gets even worse when we're asked to eat random weeds from the edge of the parking lot instead of cereal because the powers–that–be misheard some "guru" talking about the miraculous recuperative abilities of mountain goats who eat nothing but weeds.

In terms of software estimates, why are we, as an industry, so far off? Is it merely incompetence or perhaps fibbing to say what the suits want to hear or are there more subtle things going on, too? I think it's exactly the same problem that my little girl has... She can see the surface of a pile of cereal in her bowl and as she pours the milk, she sees the milk run off the surface and disappear, by the time she sees the milk level come up towards the surface of the cereal, it's too late.

Now, some folks might say to eat more, smaller bowls so you don't run into that milk hiding problem (which is related to the burning your mouth on the hot sauce hiding under the cool layer of cheese on your pizza :-). Others might advocate pouring some amount of milk into the bowl first and then pour the cereal to match. Anal-retentive types might do a myriad of studies (i.e., waste a lot of milk and cereal) to figure out some magical metric by which to precisely predict the proper amount of milk to use for the average cereal according to the preferences of some imaginary average person.

Me, I like an adaptive approach: Pour some cereal, pour some milk, mix it all up, and taste it. If there's too much milk, add a small amount of cereal. If there's too much cereal, pour a wee bit more milk. If it's exactly right, look for the Candid Camera folks (especially if I used up the last of the cereal and milk simultaneously :-). Gee, for some reason, this tastes the same as another process.

Bon apetit!

Successive Embellishment

Growing up, one of the things that I was taught was that embellishing was wrong. That was confusing to me since the actual definition of embellish is: "To make beautiful, as by ornamentation; decorate." Of course, my mom and various "teachers" really meant to teach me that telling lies is a Bad Thing(tm). Alas, like so many of us, precision in language isn't much of a priority — we over-rely upon the communication of our emotionalism to impart our intent.

In Spiral Learning, Kathy Sierra talks about the need to and benefits of iterating quickly through the entire cycle of learning. If I may be so bold, what she's talking about is the notion of Successive Embellishment. Basically, start small and simple and then iterate. But, rather than the traditional view of us following the spiral inward, tighter and tighter, we're going the other way: we follow the spiral outward, embellishing what we've already done — not only makes it more beautiful but also more valuable.

Your homework, should you accept it, is to connect Successive Embellishment with the notion of Passionate Curiousity.

It's about about the language

The Furious Purpose blog entry,Mock roles, not objects mentions a paper given at the last OOPSLA on how to think about mocking in terms of roles that need to be fulfilled. Partly, the blog entry is about the benefits of TDD (Test-Driven Development). If you're not hip to TDD yet, check out Kent Beck's book Test Driven Development: By Example.

In my mind, however, there's something a bit more fundamental and interesting underlying this discussion... That is, answering the question: what's the language that is being created? In the Furious Purpose example, it's a language that is manifested as a programmatic API in Java starting with a user login story. As programmers, that fact that we are creating and manipulating languages seems about as obvious as water is to a fish but that is precisely why I think it's something worth thinking about -- we are so easily caught up in the leaves that we miss the trees, the forest, and the bulldozer that is about to run us over.

Keys to Debugging

My buddy Terence Parr just published an article on developerWorks called Learn the Essentials of Debugging. In it, he brings up a number of essential debugging facets: Reproducibility, Reduction, Deduction, Experimentation, Experience, and Tenacity. Those six are certainly important but they aren't the whole story.

The first one that I'll add (based on the classic, nagging wife story... :-) is: Ask for help! Tom did what he could to figure out the problem and then he asked for help. Unfortunately, the boys forgot to ask me (an email expert that they know) and wasted a lot of their time -- the quoted-printable issue was the first thing that came to my mind. Doh!

However, like any problem solving process, the most crucial aspect of debugging is a more subtle issue... Admit that there is a problem. Now, obviously, Tom certainly realized there was a problem because he was actually testing his code and paying attention to the results. But, look around you. Have you noticed how many "issues" are missed, denied, and otherwise ignored? Look at the various bug databases (like the Java BugParade) where real, serious issues are ignored or outright denied for years. And, heck, at least someone bothered to recognize those problems and report them. Think about how many problems are completely missed because so many developers do only lackadaisical testing (let alone documentation or design work). Even more insidious are the higher-level dysfunctions such as the head-in-the-sand denials that I pointed out in Anatomy of Insanity?.

Humane development processes attempt to address this issue in various specific ways such as pair programming, code reviews, test-driven development, etc. The fundamental technique is increasing the cyclic rate of feedback at each level so that we will get used to listening to the feedback and then act in response to what we learned.

Rhythms in Software Development

In Fartlek - Increasing your Sustainable Pace, Erik Meade uses the fartlek concept to talk about sustainable pace in software development. However, the notion of fartlek comes from training e.g., runners. That is, the combination of short, intense work interspersed with longer, lower intensity work increases an athlete's ability to perform both in terms of their base level and their peak performance.

So, in terms of training software developers to become more proficient, there is some validity in intensive learning situations so as to point out the need to improve our ways of doing things. However, the fartlek metaphor doesn't really work in the software development world because the stress of high pressure work doesn't e.g, make us more capable of sustaining a faster base pace or increase our peak performance. The facts are clear that the stress induces worse results and more work (due to the need to e.g., fix the bugs introduced as a consequence of trying to ameliorate the stress) both in the short term and over time.

A more appropriate metaphor for what Erik is talking about is the notion of rhythm. Humans, both individuals and groups, function in rhythms. The rhythms come in various granularities such as daily, weekly, seasonally, etc. and various types such as mental, physical, emotional, and spiritual. In my experience, software developers and managers (and the myriad, inhumane "methodologies" that are used) not only tend to ignore the rhythms in our lives but actively fight against them.

For example, one key aspect in the perennial war between computer "languages for dumb people" and "languages for smart people" is how the language designers choose to either put various kinds of straitjackets on people or give folks plenty of rope to hang themselves with. :-)

The key manifestation of rhythm in agile practices is, IMHO, actually the notion of (very) short cycles, both individually in terms of individual task management (via, e.g., TDD) and group project management (i.e., XP's "iterations" and Scrum's "sprints"). The rapid cycling provides the hooks, if you will, of perceiving the reality of what's been accomplished (or not) and choosing how to adjust moving forward.

I leave it, for now, as an exercise for the reader to delve into how the notion of rhythm fits into the software itself and the systems that we create with that software.

Is it stew, yet?

Before directing you to an essay on Language Oriented Programming, I want to make one other comment about this particular pumpkin soup. We're eating it today and yet I made it Tuesday night. I learned this at a restaurant I worked in. The chef served "yesterday's soup" instead of "the soup of the day". He said that most people know that soup always tastes better the next day so that's when he serves it. This gets to John Mitchell's Slack post. It would be nice to have the time to work ahead on some project and let it sit. You could then come back to it in a couple of days and have the fresh perspective to improve it before serving.

In XP, one of the benefits of pair programming is the difference of perspectives between the current driver and the copilot. Another refreshing perspective change comes when shifting between the two roles. One of the biggest benefits of test-driven development is the forced change in perspective as you shift back and forth between test writing and code writing.

If you're not working in a pair, you can simulate this yourself by working on a task for awhile and then put it aside and come back to it later.

Obviously, the relatively short granularities on the order of minutes are good for getting some kinds of perspective, it's critical to also sprinkle in the larger granularities of days and weeks. There's nothing like a sleep cycle or three to let your brain disengage consciously but continue percolating on it unconsciously to brew some fresh thinking.

Thanksgiving, Reuse, and Slack

In Leftovers, Dan "Superman" Steinberg brings up the question of how to deal with reuse in this age of agile, lean development.

One trick is to just have a really good memory. Alas, given that most of us are human, that doesn't seem to work too well in practice. :-) Given that we're tool users and creators, we do have some options in helping us remember. A classic is a catalog of index cards -- the post-modern equivalent: a big junkyard of code (and you do, of course, keep *all* of your code in a repository, right?) and a local search engine. Another trick is creating and using an orphaned code wiki -- i.e., if the code is something interesting enough to save, add it to a wiki.

Dan asked "With coding - how do we think of reuse? Are we at a point where we start planning for our code leftovers? XP teaches us not to code for situations we don't yet need, but a customer could have a reusability story." Basically, the question is where does "reuse" fit in with the core computer science notions of necessary and sufficient? IMHO, the missing key is the notion of slack. Is there enough slack in the system to allow it to function reliably and robustly in the face of change? Most developers are exposed to this notion in practice in dealing with performance but slack is crucial in all aspects of software. To continue with Dan's food analogy, the orphaned code wiki, for example, is akin to putting leftovers in the freezer.

Big thanks to Dan for his tireless efforts in making the world a better place.

SubEthaEdit turns 2.0: Is that a good thing?

For the most part, SubEthaEdit is just a tidy little editor that runs only on Mac OS X. However, it's claim to fame is the fact that it supports concurrent editing of documents by multiple people. SubEthaEdit is a fascinating tool for collaborative editing things like conference notes or, heaven forbid, source code (i.e., pair programming where you don't have to strain your neck peering over each other's shoulders :-).

In the v1.0 days, the license was basically donation-ware and there was a clear statement in the FAQ that talked about them cleaning up the code to release it as "open source". Alas, in this v2.0 release, the license has gotten more pointedly commercial (with a non-commercial/demo/trial exclusion) and all mention of any open-source take has disappeared. Now, it's their code and they can do what they want with it but I must confess that I feel misled.

To add insult to injury, v2.0 has switched to a completely different protocol (based on BEEP, blech) and there's no interoperability between v1.0 and v2.0. What are they thinking?

Do you floss every day?

In Testing MVC actions, mock objects and code coverage, Simon Brown wonders aloud about how to think about the issue of code coverage via tests. It's quite simple really... The rationale for testing and high test-coverage rates is exactly the same as for brushing your teeth and flossing everyday.

Are debuggers a wasteful timesink?

Bob Martin starts a raucous discussion when he states that Debuggers are wasteful Timesinks.

To paraphrase a character from the last Matrix movie... Debuggers are just a tool. It is how the tool is used that is good or bad.

This work is licensed under a Creative Commons License.