The Catastrophe Cycle
Like Joshua Marinacci How do you develop?, a number of programmers, including myself, have adopted a fairly iterative development style. For the most part, this has been a conscious decision. However, there are times when this isn't always so. Occasionally, the style has more to do with the nature of the work than with the desires of the developer.
I was recently preparing for a short presentation when I came across a paper* with a couple of graphs that really caught my eye. The paper had to do with the requirements gathering process, and how it can be monitored and managed. Tthe first graph that caught my eye described a straight-forward linear relationship between the time spent gathering requirements and mastering the complexities of a system. In other words, the more time you spend gathering, the more you understand. This certainly seems intuitive, and it is how most of us model the process in our heads. Unfortunately, the model has little to do with reality. What the researchers actually found is reflected in the graph below.
Essentially, there are periodic breakdowns in the requirements gathering process. The researchers call these breakdowns, "catastrophe cycles", and largely attributed them to a form of information or complexity overload. That is, requirements engineers apparently gather until they go catatonic over the details. When they reach this point, they need to sit back and digest before they are ready for the next round. Slippage is inevitable, and there is a sense of three-steps-forward, one-step-back to it all. However, what I found paricularly interesting besides the discountinuities themselves was the researchers' belief that any requirements gathering process that doesn't experience these catastrophic breakdowns may actually be in trouble. There is a strong implication that it is impossible to have one without the other.
I don't believe it takes much imagination to apply a similar understanding to most phases of development. We've all experienced catastrophic breakdowns in our projects, and they haven't been limited to just the reguirements phase. Applying time to mastering complexity works. It just doesn't work as smoothly as we expect.
Perhaps, I should rephrase my earlier statement about adopting an iterative style as a conscious choice. Maybe it would be truer to say, at least in my case, that I've come to realize that development isn't a straight, upbroken line from start to finish, and that trying to make it so isn't worth the effort. Like Dr. Strangelove who came to love the bomb, I've come to love the catastrophe cycle. I've come to accept an iterative style not only as a personal choice, but also as a reflection of the nature of the development process itself.
*Supporting and Monitoring the Creativity of IS Personnel during the Requirements Enginerring Process", Nguyen, L., Carroll, J., and Swatman P.A., Proceedings of the 33rd Hawaii International Conference on System Sciences, 2000.