John Reynolds's Blog

What do programmers really need to know?

"...there's nothing hard enough about Java to really weed out the programmers without the part of the brain that does pointers or recursion..."

"But beyond the prima-facie importance of pointers and recursion, their real value is that building big systems requires the kind of mental flexibility you get from learning about them, and the mental aptitude you need to avoid being weeded out of the courses in which they are taught. Pointers and recursion require a certain ability to reason, to think in abstractions, and, most importantly, to view a problem at several levels of abstraction simultaneously. And thus, the ability to understand pointers and recursion is directly correlated with the ability to be a great programmer."

There's a lot of lousy software "out there".

A lot of software "out there" is great, but it contains pervasive security flaws.

A lot of software "out there" is great, but it is way too expensive to maintain.

• Flaw in Microsoft's Windows Meta File Vulnerability in Graphics Rendering Engine Could Allow Remote Code Execution announced December 28th, 2005
• Microsoft Internet Explorer PNG Image Rendering Buffer Overflow Vulnerability released June 16, 2005.
• Firefox and Mozilla IDN buffer overflow security issue disclosed on September 8th, 2005.
• Annual software maintenance cost in USA has been estimated to be more than $70 billion.
• The Cyber Security Bulletin 2005 Summary lists 5,198 reported security vulnerabilities last year.

Maybe mastering a "hard" programming language isn't much of a factor after all.

What do programmers really need to know in order to produce exceptional software?

I agree with Joel that the abilility to "to view a problem at several levels of abstraction simultaneously" is very important... But...

I think that really exceptional programmers have the ability to focus on concrete requirements. Programs are written to solve concrete problems. Abstractions have a way of taking on a life of their own, and that can lead to software that is hard to maintain when a "concrete" requirement changes.

I'll admit that it is a stretch, but even "buffer overflow" problems can be characterized a symptom of misunderstanding a basic concrete requirement (the program should gracefully handle unexpected input).

In some instances, the overhead associated with bounds checking allocated memory is unacceptable, but it is hard to see how a browser would need performance optimized to such a level.

One way to keep focused on the concrete requirements is to test compliance with those requirements all throughout the development life cycle. If you start with a test plan, you'll focus on passing the test.

The great programmer may be the guy who is most adept at writing the "right" test plan (You don't want to waste time passing meaningless tests).

Maybe teaching Test Driven Development is as important as teaching pointers and recursion?

What do you think programmers need to know?

Comments

• Nice response. I admit I had a rather furrowed brow as I read Joel's article. I understand what he's complaining about, but I think it leads to the wrong conclusion. Languages like Java were introduced to save programmers from the buffer overruns, pointer errors, memory leaks etc. so that we could build more stable software, more quickly. This is a good thing. Programming today is not the same as it was 20 years ago. To "weed out the programmers" you need to set different kinds of problems - the ones encountered by coders [b]today[/b]. To solve these problems you need the qualities that Joel is talking about. This will separate the wheat from the chaff. Just because someone is good at pointers it doesn't mean that they'll build stable, readable (!) and a maintainable framework.

  Posted by: archangel on January 05, 2006 at 05:42 AM

• The question "What to programmers need to know" is an interesting question, but I don't think it's a valid one. There is certainly an 'it' factor in this industry; my collegues often say that one individual or another has 'it'. In this context, 'it' is really defined by one key thing: enthusiasm. I've seen all kinds of different kinds of thinkers as I've floated through the consulting space in this Information Technology Community - and while some were more creative right brained thinkers, and others were more scientific, mathematic-style proof oriented thinkers, the thing that bound them together, assuming they were successful, was their enthusiasm for the trade. So my question would be "What do programmers need to have" - and I think the answer would be "Unyielding enthusiasm". Enthusiasm, irrespective of your mental structure, will drive you to learn more, look at things in a different light, communicate with other developers more (communication is key), and generally 'be on top of the ball'. The key to developing great software is not inlaid in some magic process or approach (note I'm not saying that these things can't help from time to time), it's in the diligence and attention to detail of an enthusiastic programmer. Anyway, that's just my two cents.

  Posted by: rjlorimer on January 05, 2006 at 06:46 AM

• Hey John, I'll post more here once I think it through more, trying to be constructive and all, but there is some talk going on in Fernando's blog on here about it the subject as well and its somewhat interesting. Check here http://weblogs.java.net/blog/flozano/archive/2006/01/how_should_you.html

  Posted by: olsonje on January 05, 2006 at 11:13 AM

• I like Joel's stuff, but he's being lazy on this. Every language supports recursion. Hell, every language has pointers, you just can't manipulate the pointers directly anymore in Java. Anything you could ask to stretch their brain with recursion and pointers you can ask with, well, recursion is the same, but pointers you could have them define arrays and manipulate a pointer into the array.
  
  Joel's just annoyed his favorite dipstick test needs to be updated. :)

  Posted by: ckessel on January 05, 2006 at 02:58 PM

• rjlorimer:

  "So my question would be "What do programmers need to have" - and I think the answer would be "Unyielding enthusiasm". Enthusiasm, irrespective of your mental structure, will drive you to learn more, look at things in a different light, communicate with other developers more (communication is key), and generally 'be on top of the ball'."

  I think that I may have to respond to your comment with another blog entry ;-)

  I grok the spirit of your statement... but in my career I have endured a lot of problems that were created by "enthusiastic" programmers.

  You should refine your statement to define "what" the programmer needs to be enthusiastic about ;-)

  --John Reynolds

  Posted by: johnreynolds on January 05, 2006 at 04:37 PM

• There's no corelation between the lousy software out there and non Java programmers. Lousy software == 'commercial and vested interests' == maintenance revenue Great Software == ' Linux/GNU/Apache/Professional Companies' because everyone gets to see and contribute as in open source or because of the committment to quality by the organization. So if it is bad in some sense then that's the prevalent quality of understanding of the contributors.

  Joel's point as I see it is - expose the fresh minds to the toughest challanges to prepare them for all the possibilities instead of removing/simplyfying them at the learning stage. In real life situations they should be able to make a informed decision to use Java or whatever that is most suitable for the problem at hand. With Java as the first/only language the doors to lateral thinking will be shut on them defeating the very purpose they are in the universities in the first place! - Vijay Phadke

  Posted by: vijayphadke on January 05, 2006 at 10:11 PM

• - Object Modelling Principles (DRY, Composition vs Inhertiance etc.) - Design Patterns (Use and abuse) - TDD - Recursion (it's still the most convenient way to solve some problems)

  Posted by: archangel on January 06, 2006 at 01:37 AM

• Vijay:

  "Great Software == ' Linux/GNU/Apache/Professional Companies' "

  I hear you, and don't really disagree.... But... If you follow the blog entry links you'll see that Firefox had similar problems to IE. Is Mozilla one of those commercial-vested interest companies?

  Also, if you check out the Cyber Security Bulletin 2005 Summary, you'll see that Linux has it's own fair share of security vulnerabilities.

  But that's really a digression from the point I was trying to make...

  I think that Joel is longing for a past that never was. Software that made it to production was not better back in the day when CS courses were tougher. Even with that rigorous training in pointers and recursion, we still wrote software with some pretty serious flaws.

  I don't think going back to teaching like we used to will work... I think that we need to start teaching stuff thta we've never really taught before (like Test Driven Development).

  --John Reynolds

  Posted by: johnreynolds on January 06, 2006 at 11:08 AM

• Test driven development is a nice idea, however not everyone has the resources for this e.g. time, energy, motivation, money for extra machines etc. I deal with this by feeding my data conversion programs huge amounts of archived live data and having comprehensive logging in the programs. I have discovered that real life users and live systems are much better at breaking software and exposing customer specification mistakes than module or organised user testing! Most of the nasty bugs I have discovered, have taken several weeks of live user data to reveal, via logged errors, and most were due to unexpected changes in the source system data format, because I was not informed about source system upgrades for weeks!

  Posted by: infernoz on January 29, 2006 at 06:26 AM

This work is licensed under a Creative Commons License.