A new metaphor for software development?

The current metaphor for building software-intensive systems is the engineering metaphor. Early software developers, looking at the magnitude of the challenge they faced, sought out a reasonable (to them) analogue to building a large, complex system. Engineering, with its successes in the Apollo Program, the Manhattan Project and the Hoover Dam, probably seemed a natural. It brought a systematic, and systematized (written down rules) for developing these systems. There was a clear chain of command – important in the early days, when the military was a big sponsor of software development. There was lots of paper and structure in an ‘engineering discipline of software’. Best of all, there was the promise of repeatable, well-understood projects. If there were failures, it would be easy to pinpoint why.

This metaphor hasn’t served us all that well. Software `engineering’ turned out to have few traits in common with bridges, roads, and spaceships. Software problems were hard to understand. Software materials were easy to come by, especially as the inexorable effects of Moore’s Law were seen. There was no clear sense of what the software should be doing. Should it do X, Y, and Z? Should Z be left until later? What about Y? Does anyone really understand how to build Y?

The best metaphor I have come across, for smaller projects in particular, is home contracting. A lot of home renovations fail; perhaps even as many as there are failed software projects. Some home contractors manage to have a clearly defined schedule, know exactly what there is to be done, and be flexible enough to manage the inevitable changes. This seems to account for the Joel Spolsky phenomenon — the ability to use a spec to build software. However, the vast majority of these projects seem to suffer from the problems of shifting requirements, incompetent craftsmen (can you be a craftsman and be incompetent?), and bad management.

Ultimately, what I think differentiates software development from other engineering disciplines is that software is often hard to define. Software is by definition work in the abstract. To understand the problem we’re solving, we need to work with models and constructs that don’t come naturally to humans. For a bridge, for example, or the Manhattan project, there are hard constraints and real-world factors that must be overcome. But once addressed, these factors are satisfied. Software, on the other hand, deals with fuzzy concepts, like payrolls, or other business processes.

  • There are a few objections to this admittedly ‘trouty’ article. One is the idea that conventional engineering projects are more successful than software-based projects. I have no evidence on this, other than a lot of bridges remain standing, and the Americans landed on the Moon. However, this might just be the survivor bias in effect (i.e. we simply don’t see the projects that failed). It would be an interesting analysis to do – certainly software projects are very well-documented. One example: the cause of the fire on Apollo 1, was investigated, and `1,407 wiring problems were corrected’. That’s a lot of bugs.
  • Maybe software is just a really new sub-discipline of engineering. For example, the first bridges probably didn’t work too well at first. This argument suggests that as software development ages, and we come to a better understanding of what works, then we will begin to derive that body of knowledge that exists in civil and mechanical engineering. However, I still think the abstractions inherent in software development will be a major hurdle to overcome. Often, just defining the problem is overwhelmingly complex.

Update (06-07-24): Came across a good essay on similar themes from Sun software designer Jim Waldo.

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s