Semantic Werks

Software research has always been a strange creature. Born in the 60s as computer pioneers realized the immense challenge software posed (where previously most efforts had centered on hardware engineering and design), software covers such a broad scope of domains and possibilities that categorizing it is like categorizing transportation conveyances. Are you building an ocean tanker? A cheap, self-propelled city vehicle? Something to destroy buildings?

That’s why most arguments about software tend to feature wagon-circling by members of particular domains. Are we engineers or craftspeople? The guy who builds websites for local veterinarians is appalled by the demands RUP places on his development process. He finds Java and Tomcat ridiculously over-engineered. At the same time, the multinational IT support person can’t fathom how anyone would develop without a fully elicited requirements model or class diagram. She looks at Ruby and sees a newfangled bastardization of functional programming and Perl.

These aren’t the only camps, either. It often seems to break down over project size, potential harm, and importance. NASA has very few defects in its code, but spends billions of dollars to get to that point. Not something 37 Signals is interested in doing.

This is why I find the term ‘software engineering’ misleading. Sure, back in the 60s, 70s, 80s, venues like ICSE were largely about giant defence projects, things like Star Wars (see here for a great rant about how futile that effort was, though). Software could be engineered, right? Throw peons at the project, a bunch of money, and with a healthy management team the project would succeed.

Since the dawn of the open-source movement, I would say, and the rise of personal software in the late 80s, research on large-scale software was increasingly less useful to a large number of practitioners. This led to a number of non-academics proposing truly innovative and successful methodologies — Scrum, FDD, XP, Lean/Kanban — because, in my view, the researchers just weren’t able to see the change coming. Partly this is because as a professor, you need secure funding, and small firms cannot provide this. Inevitably you end up working on Oracle instead of MySQL, for example.

The exciting things is that doing research in software fascinates me precisely because of this diversity. But that research must be targeted appropriately. Don’t insist on the primacy of UML for every possible project — it just doesn’t make sense. Don’t see statements like “people over process” and assume that process is irrelevant.

I think in order to more fully embrace this awesome scope, to make research (more) relevant, we ought to stop pretending we all do engineering the way they had hoped — although not all of them — at the NATO conference in 1968. We need to more carefully identify the relevance of humans in the software loop. We need to accept the inevitability of change and inconsistency in software and organizations. We need to figure out what’s useful for the ten-person company, as well as the thousand person company. We ought to just call ourselves software scientists, and go from there.

Currently, it seems to me that practitioners of all stripes get their information thusly: first, via colleagues or anecdote; secondly, from websites like Slashdot or Digg; thirdly, from vendors and PR agencies; fourthly, from industrial research firms like Gartner or Forrester; and finally, lastly, from academic research papers. I would really like to move academic research up this list.

Vincenti, Walter, 1990. “What Engineers Know and How they Know It: analytical studies from aeronautical history”. Johns Hopkins University Press.

At RE08 in Barcelona, Michael Jackson mentioned this book as a must-read for people in requirements engineering, and respecting my elders, I dutifully picked it up.

The first contribution of this book (most important?) is to highlight that engineering (and I sort of unthinkingly lump software engineering in there, for the moment) is NOT just the application of scientific principles to real-world problems. They are ‘separate spheres of knowledge’.  Anyone who has tried to construct the most rudimentary technology — e.g., a tree-fort, change a tire, etc. can verify this. While on paper the solution is obvious, in harsh light of day, the nail doesn’t go in deep enough, the lugnuts are seized, and so on.

I found the book often went into excessive detail, of interest only to aeronautic buffs, but there were sections at the beginning and conclusion of each chapter that were highly interesting. Perhaps most relevant to my research, and to software engineering, was the chapter on aircraft qualities. Qualities include aspects like how ‘heavy’ the control stick is, how stable the plane flies, etc. Anyone who has driven several different types of car can relate to these properties, I would think: does the car shimmy at high speeds (like my first car, an 89 Toyota Tercel)? Do the brakes feel squishy? Does the door have a convincing thunk when closing, or a tinny clang?

For early aircraft design, these qualities were secondary to functional aspects: ‘With the essential proviso, of course, that an airplane can be reasonably and safely flown by the pilot, its utility depends crucially on these other characteristics [speed, range, ceiling, carrying capacity ...]‘. He goes on to add that only when these functional properties were largely satisfied (and here I assume he means that there were diminishing returns on further research into these functional characteristics) was flying quality considered. Eventually, specs for new aircraft came to include sections on flying qualities, encapsulating ‘the notion that desired subjective perceptions of pilots could be attained though objective specifications for designers’. Because these qualities were subjective, the specs tended to be design guides to assess tradeoffs rather than objective, measurable requirements (much like in NFRs and software design).

Vincenti argues that engineering knowledge grows via an evolutionary variation-selection process; as a result, ‘technology’s failures require examination as well as its successes’. The important lesson is that approaches that attempt to design something perfectly from the beginning seem doomed to failure. The best we can do is prepare to throw one version away, until we have a good body of evidence to support one particular solution. For example, NASA tries to re-use Mars expedition software and hardware as much as possible; e.g., the particular circuit board, the software language, and so on. The reason for this, as Jackson explained in his keynote, is to leverage the benefits from “Normal design” and specialization.The characteristics are known and well-tested; keeping these things constant allows for incremental specialization at the margins (e.g., improved science packages, new rocket engine, etc.)

My take-away is that the most important thing to understand in approaching a new problem is what pieces of that problem are normal and what pieces radical. Focusing on the radical is key, because we can assume this is where the failures and iteration will need to happen.

Julio Leite reported on UML usage statistics, showing penetration of UML is ~23%. In an upcoming publication, Jorge Aranda surveyed small-sized software firms and found their use of modeling to be quite low, and typically of the UMLAsSketch variety. So…

Q. Why is UML so little-used, despite the fairly heavy-duty marketing and machinery behind it?

A. The long tail. See the graph below.

My contention is that of software development teams, the vast majority are working on less complex projects. For example, a month-long website redesign, a custom CRM extension, etc., etc. There are comparatively fewer working on multi-month, disruptive software systems.

Some questions, other than where can we validate such a claim.

1. How can we measure complexity? Time to completion? Size of written source code? Platforms targeted? Risk potential? Project budget? I would probably lean to project budget as a very rough guide — we all use dollars, and the cost usually reflects an a priori assumption about complexity.
2. What sorts of tools would the mushy middle of this tail need? I.e., those firms doing median complexity projects? Clearly the companies at the head of the curve don’t need the Thor’s hammer of UML or DOORS when the Owen’s* hammer of technologies like Excel do just as well. But there must be a tipping point where these technologies start to get more and more useful and important. But
3. Is there good empirical evidence that even the long-tail denizens need UML, DOORS, or Rational? Indeed, we might even expect that such large-scale firms would customize requirements management, configuration management and other helper technologies for their own purposes. This seems to be the case for Linux (GIT), NASA (SCR, formal methods, etc.), and probably others.

* Thor’s weaker and lesser-known half brother, an accountant in Little Wopping, England.