Semantic Werks

In ObservedRequirement, Martin Fowler makes the following comment: “the word ‘requirement’ is inherently waterfallish” (in relation to a quote about BRUF). A lot of these claims are unsupported by rigorous empirical study: either they refer to closed-data, proprietary reports (Standish), or base their experiences on anecdote. Does agile scale to projects that have more than 10 people involved and exist for several years? Could the Air Force be agile? We don’t have very good answers to these questions, yet. Nonetheless, the ‘agile’ movement has a lot of experienced, seasoned developers and consultants behind it, and it would be foolish to assume it was irrelevant because the data wasn’t there.

Fowler’s solution to adaptive software is to study what people are doing, and update the system accordingly. There are two problems with this. One, it assumes the software you build has people as direct users. Plenty of software is only indirectly used by people; for example, software to connect wireless modems, software to run microwaves. In general, the software Fowler seems to be talking about is one niche in the entire ecology: admittedly a big and sexy one. The second problem is that it falls into the task-artifact cycle [1], that is, it will only reveal those problems made manifest by that artifact. Logs will never be able to show what problems people had that were caused by missing features, or those things which they wished they could do.

If we want to build evolving software, this approach would be useful to tell us what bottlenecks there were, what paths people took through our software, and so on. But presumably we would have to build this agility into the tool to begin with: the monitoring framework, the reporting functionality, the self-* systems autonomous computing talks about. In this case, aren’t we now doing stable software engineering? It may be cumbersome to elicit all the requirements at once, and likely not even possible, but perhaps what people like Suzanne Robertson are arguing for is to try and get as many as possible early on. If someone tells us to do something, do we not want a sense of the purpose, a teleological explanation? The knowledge to help motivate our efforts, that even if we start by digging holes in an empty field, we may be building the Taj Mahal?

[1] See Vicente, Cognitive Work Analysis, p. 103-104. The term is often used positively, in the sense that good iterative design can accommodate the changing requirements involved. However, Vicente makes the case for escaping the tyranny imposed by the initial design: “Prototyping and usability testing could iterate an existing design, but couldn’t suggest wholly new directions (Beyer and Holtzblatt).

Martin Fowler, respected senior figure in software engineering practice, has a post describing ‘schools’ of software development. A school in this sense is a group of like-minded individuals who subscribe to common philosophies. For example, Fowler describes himself as belonging to the object-oriented, agile-practitioner school (although this is necessarily vague).

I like this notion, particularly because it seems to model the idea of software as craft. That is, software development is somewhere between art and engineering. Other crafts might include cabinetry, armor-making, carpentry, and so on.

Fowler’s post, however, seems to imply that all schools (or at least those that have some basis in practice and can enumerate their shared beliefs) are equally valid. As an aspiring software researcher, I would disagree (note that my response isn’t grounded in any empirical experience). My feeling is that in fact, there are better sets of practice (perhaps for certain situations) and worse ones.

Putting aside those schools that are clearly created merely to serve the needs of a particular vendor, it is my fervent hope that at some point, we can define measures of success that will enable us to say where, and when, a particular approach is useful. For example, is static, type-checked development always worse than interpreted, loosely-typed development? Is test-driven development always best? When is it useful?

My sense of reading the reports of practitioners, and the research literature and history of the field, is that we still do not have a good sense of how to even ask these questions, let alone answer them. For example, Fowler references a post of his, “We Cannot Measure Productivity“, wherein he makes the point that it is (impossible? difficult?) to measure how productive a programmer or team is. I agree. But still, that doesn’t mean that there isn’t the good old Gaussian (power-law?) distribution behind a series of software projects. Namely, there are good projects and bad ones, and a bunch of mediocre, ‘met expectations’ projects in the middle. So presumably there is some way to measure these outcomes.

I think there’s a hint to an approach to these issues — specifying measures of success, productivity, etc. — in Fowler’s statement that ”any true measure of software development productivity must be based on delivered business value”. Business value may be an even more vague concept, but at least there’s a starting point. I would apply the same standard to software schools. Your school is superior, in some context, if it can deliver more value to the customer than a competing methodology. While Fowler is pessimistic on our ever being able to accurately measure business value, I’m more naive. I think we can, and should try.