Semantic Werks

Big Requirements Up Front are certainly not in favour with the industry thought leaders in software development. But I think the idea of specifying your requirements ahead of time, as Parnas says in “A Rational Design Process and How To Fake It” is worthwhile.

Consider the case of climate models. Or smaller open-source projects. Here the developers ‘scratch their own itch’ and the requirements are almost always implicit. What to develop next is driven by an internal process, where the Customer (in the XP sense) is always present by definition, since the developer and the Customer are always the same.

And yet these projects often run into trouble because the requirements are not explicitly modeled, where by ‘modeled’ I mean somehow written down and discussed, prioritized, etc. (ed.: Justify this claim!). Backwards compatibility with previous instances is not maintained, reliability is awful, effort is duplicated, and so on. I think only in projects that are being done the same as last time – cf. Aranda et al, 2007 — is there a hope of totally ignoring requirements explicitly.

Why did we start with the BRUF paradigm in the first place? I think it was because like everything in software development, (and computing itself for that matter) things started with the US Defence Department. And their needs are so different than those of a small software development company like 37 Signals that it’s a totally different animal. You have multiple competing vendors, huge safety considerations, multi-year and multi-billion dollar developments. It would be foolish not to do some up-front design. That’s not to say that iterative, frequent release models can’t work in the Army – I think they could. But a little up front planning could save a lot of time designing things you won’t need.

Because the army gives so many dollars in grants, the prevailing academic attitudes are concerned with defence sized issues. The problem is that these concerns are not shared by other sectors. And so small “a” agile started as a reaction to this top-down design.

I’m not defending the IEEE requirements template, or the amount of documentation for CMMI certification. But I very much believe that some form of explicit requirements modeling is important for projects. Look at Scrum — most people are doing user stories with story points. These are requirements models! Simple, but still explicit, prioritized, costed requirements. And people like Scott Ambler argue that to scale Agile, you need a little more – you need ways to organize the stories, to assign responsibility, to manage the stories.

The reality is that when systems get more complex, for most of us it is impossible to keep track of the scope and scale of the system (unless you are Linus Torvalds, perhaps). In that case, to communicate with the rest of the team you will need something to share — and requirements neatly capture what needs doing. I think the word ‘requirements’ itself has come to mean BRUF requirements, when it properly means the set of new properties of the environment your new machine will bring about (Jacksonian requirements (pdf)). Under this definition requirements becomes a much broader term.

One of the big problems is that we understand things in this industry using anecdote. And so many horrible anecdotes have come from “big requirements up front” projects. But in those cases, any methodology would likely fail if the organization lacked maturity. And conversely, any mature organization could use any methodology it chose and succeed. Those organizations know how to motivate people to get excellent work and know not to change scope midway, not to alter budgets, and so on. I think these factors are ultimately much more important than the particular development methodology one chooses.

One of the central concepts in various agile software development techniques is the user story. Why go with a ‘user story’ instead of a rigorous requirements elicitation, culminating in a model of varying complexity? Is a story enough?

What is a ‘user story’?  These statements are (originally) from Agile Model Driven Development with UML2. Caution: as far as I can tell none of these statements is  empirically tested (or possibly even derived).

• “a user story must be able to be implemented by two people in a single iteration/cycle”
• “the user story card includes an indication of the priority”
• “You often create a stack of user stories during cycle 0 to identify the scope of your system”
• “Stories make it easier to divide the work into tasks than use cases” p 321
• “user stories are malleable and change during cycles”
• “because they’re small that they are very easy to work with and to evolve”
• “user stories contain so little information you will need to flesh them out a bit when you first work with them”
• “User stories are merely the starting point” for figuring out how to implement these stories.

Here’s an example, from the same source: “Parking passes can be paid via credit cards”.

So this user story takes two people pair programming about a week (for an average project). Really? Anyone who has worked on task-decomposition, whether in personal life or for work, would probably agree that the tough part isn’t determining goals, but figuring out how to achieve them. In other words, what concrete set of steps do I take to get this achieved? Presumably going from the story to the implementation is something a competent developer can handle, but it seems to me that a lot of design is hidden within that story: what does payment mean? What types of credit cards? Do we need to handle monthly and daily parking passes? And so on.

I would like to see a distinction drawn in talking about software development: We can talk about the process used to develop software: e.g., plan-driven, design up front, vs. work-in-progress limited lean development. But we can also talk about the artifacts used to drive the development. We could use stories and an onsite Customer. We could also use more formal requirements models, issue trackers, bug repositories, etc. I think these two are to some extent orthogonal. Nothing says the model must be set in stone before implementation.

As part of my research, I try to keep up with the trends and tools in software development (it has nothing to do with procrastination, either).

I’m beginning to wonder if maybe I’m missing a large chunk of people in my reading. According to the blogs and sites I follow, agile is far superior to other methodologies, Ruby and Python are orders of magnitude more productive, REST architectures are preferred, and, importantly for me in particular, requirements are best understood as user stories, and not as lists of MUST and SHOULD items. Let’s call this the agile world.

But I feel like there’s this other world, the world I hear about when I read academic papers, or read IBM/Oracle/MSFT blogs, or hear about work at large companies. This is the world of J2EE, of COBOL, of UML models, of reams of paper documentation, of Rational Rose, of ESB, of SOA, of WS-*. This, for lack of a better term, is the waterfall world.

Is it just me, or is this waterfall world not getting the message that people like David Anderson and Martin Fowler are spreading? Is there a reason for this? Frequently argued is the point that these systems are orders of magnitude more complex, older, and larger, and therefore ‘agile’ methodologies just won’t work. And true, some of the agile proponents are tiresome ideologues who talk but don’t listen.

So while it is appealing to self-identify as the little boy pointing out the emperor’s lack of apparel, maybe I’m out of touch. Maybe there really is a need for complex modeling initiatives, for up-front requirements documentation. I’m not talking about safety-critical software either — that’s a whole different kettle of fish, and a straw-man for the purposes of this discussion. No, I’m specifically referring to large corporate IT departments, process control software, line automation, etc.

I guess I just don’t know how to experience the waterfall world without being involved in it first-hand. In the agile world, there is hype aplenty — hype sells books and seminars and certifications — but , at the risk of sounding repetitive, experience reports and anecdote just won’t cut it (for either side). It’s extraordinarily difficult in the software community to get at any objective source of information — except surveys, typically.

So who’s up for a PhD establishing the benefits of agile and its ecosystem over waterfall and its suite of tools? Can such a thing even be established?

I just watched an excellent presentation by David Anderson at the Agile 08 conference. He talked about moving agile to more complex, enterprise-scale projects, and how the current agile practices work.

My big take-away was his characterization of requirements as ‘perishable’. In his view, unmet requirements are like unsold inventory, essentially liabilities until they can be turned into working systems. He borrows from the just-in-time philosophy of Lean manufacturing to emphasize how unsatisfied requirements ought to be taken off the shelf and implemented as soon as possible.

In his view, future software/system architecture decisions will be much higher-level, business-value decisions. He mentioned software product lines as an example: the architect is someone who can separate product variability and encapsulate common behaviour, rather than over-specifying lower-level designs. There will be an increased need for ‘customer intimacy’ to generate unique systems that capture a core competency or competitive advantage.

There’s a lot more in this talk, including some interesting ideas on community evolution, so it’s well worth the time to watch.

Having returned from the Requirements Engineering conference in Barcelona, I’ve been pondering where my own research fits with the talks I listened to. One of my concerns, perhaps needless, is the relevance of my research to practice (both current and potential). The RE conference has parallel streams of industrial reports and academic papers, to emphasize the purpose of research, I suppose (although other fields don’t seem as concerned as we are with relevance).

Many of the tools I’ve worked on are fairly heavyweight, and assume a fairly large degree of co-operation from the user regarding his/her commitment to the modeling process. But is such a tool important for most users?

Siemens, for example, presents numerous reports each year at the conference arguing for heavy process around requirements engineering, but since it is the RE department presenting the research, it’s hard to tell how much these tools and methodologies are appreciated internally.

Microsoft, apparently, uses documentation extensively in development (security analysis, i18n analysis, performance analysis, etc), but one might argue that this is hardly an endorsement. I’m not familiar with what Google tries, but my sense is that they are much more about the competitive environment for new ideas (and have many fewer challenges with respect to legacy systems support).

Cringely has a good post about organizational support for innovation. In it, he quotes the following: “You wouldn’t need “change management” if you made continuous improvements at the functional level the responsibility of every individual and team cluster (Janna Raye),” So it would seem that in the industrial context, there is a big difference in how, and what, requirements practices are used.

The questions that one must consider is clearly one of future directions. Jorge Aranda‘s paper suggests that for many small and medium-sized companies, at least, the use of requirements models (and associated tooling) will be (is) negligible. The business analysts, who are probably also programmers, generally understand the environment very well, and tweak their product to accommodate new clients, rather than doing revolutionary design.

Greg Wilson characterizes one dimension of software systems according to how agile or stable they are. An agile system to him is one that tries to duck the punches of unanticipated evolutionary changes; a stable system tries to build itself so solidly that these changes cannot affect it.

Martin Fowler, in turn, characterizes these as evolving or planned designs. Clearly for planned designs tools and models become more important (though arguably still not necessary). Evolving designs focus more on working code, code as documentation, and testing as the way to verify the match between requirements and implementation.

A colleague of Fowler’s at Thoughtworks proposes Consumer-driven contracts, essentially user stories for web services: “To recognise the specific benefits and outcomes supported by a service, we need to understand that service in its collaborative context.” How they help with change management: “consumer-driven contracts help contextualise compatibility issues based on extant obligations and relationships. A change to a mandatory element need not always be considered a breaking change; rather, changes that require some form of versioning can be identified with reference to the consumer contracts they break. If existing consumers aren’t using a mandatory message element, then changing or removing that element need no longer be considered a breaking change.”

This turns the burden of change management over to the client, requiring him/her to determine his/her needs, and then managing the change in that context. Previously, a new release from, for example, Oracle, required a fairly binary decision from the IT department: thumbs up or thumbs down on the new product update. In the service world, the promise is that such monolithic upgrades will be a thing of the past.

The requirements problem doesn’t disappear, of course. Indeed, a consumer-driven contract is a form of customer requirement, which should be (largely) satisfied to build a successful project. What is changing is the form these requirements take: rather than large, static, possibly stale models, the requirements might take the form of acceptance tests, behavioural checks, or quality of service policies.

In terms of my research, which focuses on unanticipated changes to a system domain, the relevant questions surround a) understanding how this type of evolution might be best characterized (yes, yet another taxonomy) and b) what type of tool support might help manage these situations. My current thinking is that while change is always unanticipated in some form, we should attempt to support processes for deciding how to adapt to that change.