John Reynolds's Blog

Bigger fish to fry

A few years ago I became the Development Manager for a small InfiniBand software startup. I had been a Team Lead for some time, but this was the first time that I was "the Boss" rather then being the "First among peers". It was a hard transition for me, and I quickly overwhelmed myself and pissed-off my developers by trying to micro-manage them. There is some question as to whether or not developers can be managed at all, but I assure you that we cannot be successfully micro-managed.

I'm not sure if I ever really got the hang of the role (the company died when InfiniBand stalled), but I did learn a lot about what works and what doesn't. Much of the credit goes to my wife Teri (the teacher) for giving me the following advice:

"Developers are problem solvers. Don't tell them what to do, tell them what the problems are."

This amazingly simple advice saved me. We are problem solvers. We aren't drones. Give us a problem to solve, and we'll be delighted to come up with a hundred ways of fixing things, and some of those solutions might even be practical.

The downside is that not all of us are particularly adept at identifying which problem we should work on. For me, the most important problem to solve is the one that I am most interested in solving a the moment. Intellectual challenge and novelty always get prioritized higher then mundane things like those nasty little exceptions that get logged in the middle of the night.

I think that the Java community and open-source developers in general are having problems in identifying which problems to work on. There seems to be a huge duplication of effort, and while folks may not be reinventing the wheel, there do seem to be an awful lot of enhancements to the wheel underway (Java.net alone is hosting more then 1000 projects). A lot of great work is being done. A lot of great ideas are being pursued. But I can't help feeling like I am running as fast as I can just to stay where I am.

I could list examples, but folks are passionate about their causes and being negative doesn't really help much. Why ruin a good beer by crying in it? The Java/Open Source ecosystem is a heck of a lot better place to work and play then the "rival" ecosystem that I escaped from, and I sure don't want to go back.

I am just a blogger. I haven't written any great books, and I haven't contributed any great software. I'm just a guy with opinions, and one of my opinions is that we need some new visions to guide us. As my father might have said: "We need bigger fish to fry".

Give us the right problems to solve, and we'll do marvelous things. Leave us to decide what to do on our own and we'll probably be arguing about the EJB 99.99 spec in the year 2027 (by which time Java developers will have eleven hundred and seven MVC frameworks to choose from). If we tackle the right problems we'll be forced to make infrastructure decisions and get on with it.

All this speculation begs the question: "Who should tell us what the important problems are that we should be working on?" I think I'll follow my wife's advice and pose that question as a problem for my readers to solve.

Comments

• The users should tell us
  For example: a new project starts up based on a neat idea. 100 people think this is a neat idea and download it, read the README, try it out, and leave because it's not ready or does not have the feature they want. No feedback email is ever sent to the author.

  This process is totally broken. The user needs to complain, whine, cry, shout, whatever. The message needs to get across and the developer needs to listen.

  We're told, "Don't complain, submit a patch". How self-defeating! This is telling people you don't want feedback. Complain! SHOUT! Vent your frustrations if you like. The feedback loop is critical and must be encouraged even at the expense of a little ego and pride.

  it's all about solving the user's problems. Encourage them to tell you what they are.

  Cheers.
  

  Posted by: markswanson on July 25, 2004 at 11:51 PM

• What should happen before the project starts up?
  Feedback once a project starts is certainly important, but what should be done before the project starts up?

  Some projects are just for the education of their owners, and those certainly don't need any external justification.

  Some projects are "evangenlical"... the founders have a neat idea that they'd like to share. Once again, not much external justification is needed.

  The third kind of project is in response to an existing project. Perhaps the founders are inspired by an existing project to make something better. This inspiration could be admiration, or it could be hatred (or anything in between). I think these projects probably need some external review before they take off running.

  Posted by: johnreynolds on July 26, 2004 at 04:11 AM

• The users should definietly tell us!
  User feedback is the most essential, and yet the scarcest element. The current options here are rather limited. I think java.net could do more to help foster it.

  Allowing user ratings of projects for example; or having a most popular downloads listing; these would go a long way toward increasing community awareness of projects that are 'catching-on'. This would further help by letting prospective project owners know two things:
  What types of projects are receiving a lot of interest.
  What types of projects do not need to be reinvented

  Expert editorial reviews, if such a thing could be mustered, would be another teriffic help. Right now I use JARS as a resource, and I also like jfind, due to their broader charter. They are both very good, and I heartily recommend them, as the community can never get too much support.

  A little subtle promotion never hurts either ;-)

  One last idea; a special forum for people to talk about what type of things are really really difficult to do in Java. These are areas either in strong need of project support, or a perfect opportunity to recommend a specific project solution.

  I believe these things could put java.net at the forefront of community support.

  Posted by: cajo on July 26, 2004 at 06:55 AM

• Standardized Project meta-data?
  Perhaps we should steal some of the meta-mail ideas and apply them to java.net projects (hopefully this would spread to sourceforge, codehas, etc.)

  Each project "description" could include searchable meta-data that would help categorize the project. Wouldn't it be nice to search on relationships to other projects, number of active commiters, most recent release, number of open bugs, etc?

  Posted by: johnreynolds on July 26, 2004 at 07:22 AM

• I agree with the other comments
  We do need user feedback.

  What people fail to realise though that there is a highly trained group of people for whom knowing what the users want is their job.

  They are the... drumroll please...

  Marketers!

  Maybe when we (as in: programming geeks in general) stop bashing marketing and the marketers, and learn how to attract them to our cause, then we will make significant progress.

  There's a reason why M$ has 7x as many marketers as engineers.

  Yes, the marketers are 'evil' in that they say that things like security aren't as important as features.

  But this isn't the marketers fault!!! Its because to most people security really isn't as big an issue to the consumers as convenience.

  Posted by: rickcarson on July 26, 2004 at 11:22 AM

• Yes, they should
  You are absolutely correct. I know, because I'm the developer of a moderately succesful open source Java project, Vector Visuals (http://vectors.sourceforge.net), which has had around 1900 downloads.

  I have, from over 400 days of the project being listed, and all those downloads, received...ready?...ONE EMAIL from a user. That email was asking for help...but then was quickly followed up by one saying he had figured it out on his own. Now, granted, in later exchanges with him I got some good user feedback, but my point is that it took more than a year just to get a single comment of ANY kind!

  This is a real problem. I don't know what we, as open source developers, can do to solve it. My email address is available, I openly invite comment on my project pages, and I have Open Discussion forums...all of which go unused.

  But the downloads keep flowing... It boggles my mind that people are so reluctant to complain. I mean come on, they'll complain about everything ELSE!
  

  Posted by: ccerberus on July 26, 2004 at 12:21 PM

• Standardized Project meta-data?
  I actually like this idea, and have been thinking the same kind of thing. I've implemented an Open Source early stab at such a thing, as part of the Thought River Commons (http://commons.sourceforge.net). Take a look at the "ContentCard" class:

  http://commons.sourceforge.net/api/com/thoughtriver/open/util/metadata/v2/ContentCard.html

  
  I know it's not just for software, but it encompasses that idea, and is a general meta-data construct. I made it a Bean so it can be externalized using the XMLEncoder/Decoder stuff from the SDK. I envisioned the XML version of an externalized ContentCard could become sort of the standard description format for different things.
  

  Posted by: ccerberus on July 26, 2004 at 12:29 PM

• Don't forget Product Managers
  I agree Rick. Marketers are very important, as are Product Managers.

  I wish the Apache Foundation would hire a few Product Managers to consolodate their Jakarta product lines... Struts, Velocity, Turbine, Beehive, etc. These are product lines, and they need to be treated as such within the company as a whole.

  Talk to customers, prioritize bugs, consolodate the offerings, simplify the message. It's what users really want.

  Posted by: johnreynolds on July 26, 2004 at 11:34 PM

• Frying the right fish I think I am seeing a common trend here, similar questions being asked, beginning to link together with each other and with questions that have lurked in my mind for a while. Obviously the drive for all this activity is that we perceive some lack of perfection. Equally obviously, all these efforts are not leading completely to the Platonic ideal, otherwise it would be recognized (hey, we can't all be stupid).
  What if the imperfection lies in the basic concepts that we are working with? Certainly things are moving in the right direction, but as long as we don't identify the Platonically pure ideas that we really want to work with, we are basically shooting in the dark.
  Objects certainly are an improvement over what was before, and by being in some sense closer to the ideal it gives a more robust foundation. Automatic garbage collection is another: why should I care that physical computers are finite when the Turing machine has an infinitely long tape?
  But objects aren't quite the ideal, IMO, we need even purer concepts to work with, so that our ideas can be expressed more perfectly. There should be underlying concepts that can be used to describe, e.g., both Lisp programming and Java programming in the same abstract terms, a mathematical purity unmarred by practical issues. What are we really trying to do when we wish to create the perfect language/framework?
  *plug*: I just requested a new project called J'explique to discuss these issues and see if by stepping back a bit to catch the right fish we might not then be able to go forward and fry the biggest fish.
  

  Posted by: tobega on June 28, 2005 at 04:29 AM

This work is licensed under a Creative Commons License.