Calvin Austin's Blog

So you want to rewrite or re-factor your application

Why Code Rewrites Don't Always Work.

Give an engineer some code and one of the first things crossing through the mind of some engineers is, "Should I rewrite or re-factor that code?". The logic being that the new version will be ultimately be better in some way than the original.

However simply rewriting or re-factoring an application doesn't always warrant the effort you put in. The rewrite version may generate new, but different bugs. As well as include some of the old existing bugs. The new version may ultimately take far longer and cost more than the original. It could even run the risk of missing its time to market or not really meet users needs. I use the term 'time to market' to cover open source projects as well.

So what should you do if you want to rewrite an application, or one of your colleagues is planning on a great rewrite project and you want to convince your manager that they shouldn't just say no?

Well, I have some tips and techniques that may help. I've been developing software for over 20 years, most recently at SpikeSource. I've seen the good and the bad and formed some of my own thoughts over the years. I'm interested in your thoughts too!

Why does an application need rewriting?

First what is a rewrite? I use the term rewrite and re-factor interchangeably although there is a difference. A rewrite normally implies additions, deletions and corrections to an application. A re-factor is a change that should leave existing interfaces unchanged. In this context both apply but more typically a re-write is where most of the issues appear.

To find out why an application needs re-writing, it really helps to find out what happened during the original development. Ultimately many projects that now require a rewrite, experienced one or both of the following.

Case 1. Short of time and was under-resourced
Case 2. The project was breaking new ground or was experimental

The results of these constraints is that in the first case the project is very likely under-tested or even untested. In the second case the project is more likely to be under-specified or unspecified, especially if the end users are unknown.

Steps to make the re-write a success

Case 1: Short of time and under-resourced

Symptom Checklist:

1) Bug fixes and hacks that clutter or make code messy
2) Bug fixes that don't have test cases
3) Documentation if it exists, doesn't match the source code.

Resolution:

The first question you should ask is this. If the original author didn't have time to complete the project what has changed, is there more time now?

Does the application have a large mature installed based? The risk/reward ratio on rewriting this type of project is going to need significantly higher development costs to mitigate those risks and make the project a success.

Were the tests hard to implement or run, for example i18N and GUI tests are often expensive and test optimizations are often made.

If you do go ahead, spend time to revisit the design and make the specification complete. Backfill tests to validate the current behavior. Those hacks may initially appear like they can be re-factored but without the tests to verify changing the code may introduce a new bug. By putting more cost into validation the rewrite has a higher chance of succeeding.

Case 2: Experimental

Symptom Checklist:

1) Specification will be incomplete, or not reflect the final product
2) Tests, if they exist do not match the end user experience as there were no end users

Resolution:

Again re-write the specification completely before starting.
Try and extrapolate test case scenarios through initial bug reports.
Finish use cases and plan to spend a larger effort on test cases before starting coding

Conclusion

Rewriting a project should never be undertaken lightly. The difference between a rewrite success or a rewrite failure can be a matter of a few bugs either way. Ultimately don't repeat the mistakes of the original author, especially if that original author was you.

Comments

• This really can't be discussed enough. It doesn't matter if you originally developed in a dynamic or static typed language, the following are essential

  1. ALL test methods/classes
  2. Scope realistically for the rewrite, if it all looks iffy, make it a fix only.

  This is definitley something Joel on Software got right: When you rewrite software, you are erasing all the knowledge and maturity of the original system; be prepared to be superior to it, or don't do it at all.

  What surprises me (in any language environment) are the amount of bone-headed developers out there that jump into re-writes armed with only smug overconfidence.

  Posted by: ilazarte on April 21, 2006 at 04:52 PM

• In my environment I feel frustrated by too much "don't fix it if it ain't broke". They never have enough time because changes are usually mandated by regulations, but there are often long in-between periods where there is time available to improve things. On the other hand, I would agree that you do need sound business reasons for rewrites, e.g.

  Is there a lot of manual work generated by the system that could be reduced?
  Can we foresee some things that will be easier to do later if we re-architect parts of the system?

  The central point, I think, is the cost of testing. If you are used to doing a five week manual test cycle, especially if the system is a giant monolith, the costs will almost always be prohibitive.

  Posted by: tobega on April 22, 2006 at 12:50 AM

• From my experience you have the following situation:

  Total crap in code.
  There are users (no sense to improve crap if there is no users).
  The CVS or other source control system exists (there is no way to do something without it).
  No Ant - no build in one step.
  No tests.
  No specifications.
  

  The solution is following (my vision):

  If you are not good enough as the software developer find the software developer.
  Write the Ant script - you will need to build the deployment package in one step.
  Use IntelliJ IDEA (I do need this tool for difficult projects). Get two monitors if possible
  Start fixing the obvious and most annoying bugs. Fix the errors in logs. During this process you can learn more about system and you can delete some crap.
  Never insert "hacks" or other crap. Stay late at work, dig the code, use logging and debugging but find the root cause of the problems.
  There is no place for automatic tests - there is no units in original systems. But always ensure manually that you fix works after each fix. Also check use cases you could broke.
  You will introduce new bugs - that's live.
  If you work with the system that requires I18N you need to migrate to Unicode and fix the bugs in the code that is not aware of the difference beetween bytes and chars.
  The database is the most important part of the system - take care about it.
  Plan the big changes - discuss it with management and users. Be ready to fix bugs fast after these changes. Try to split big changes to smaller ones.
  

  Posted by: yasha on April 23, 2006 at 12:40 PM

• Great comments. Joel basically says never re-write, which is one extreme, the other of course is to always rewrite. I think there is some middle ground which prompted by blog. I think the success of a rewrite is to treat software as an engineering discipline not an artistic one.

  
  Let me explain. In the artist domain, eg writing, painting, composing, all successful works have to be rewritten and re-edited to an extensive degree with a fuzzy completion criteria. In engineering, re-engineering /re-writes are a carefully planned operation, to improve and make more efficient.

  Tackling a re-write should be more akin to re-building a house or road than getting out a best selling novel.
  

  Posted by: calvinaustin on April 26, 2006 at 11:57 AM

This work is licensed under a Creative Commons License.