Semantic Werks

As part of my research, I try to keep up with the trends and tools in software development (it has nothing to do with procrastination, either).

I’m beginning to wonder if maybe I’m missing a large chunk of people in my reading. According to the blogs and sites I follow, agile is far superior to other methodologies, Ruby and Python are orders of magnitude more productive, REST architectures are preferred, and, importantly for me in particular, requirements are best understood as user stories, and not as lists of MUST and SHOULD items. Let’s call this the agile world.

But I feel like there’s this other world, the world I hear about when I read academic papers, or read IBM/Oracle/MSFT blogs, or hear about work at large companies. This is the world of J2EE, of COBOL, of UML models, of reams of paper documentation, of Rational Rose, of ESB, of SOA, of WS-*. This, for lack of a better term, is the waterfall world.

Is it just me, or is this waterfall world not getting the message that people like David Anderson and Martin Fowler are spreading? Is there a reason for this? Frequently argued is the point that these systems are orders of magnitude more complex, older, and larger, and therefore ‘agile’ methodologies just won’t work. And true, some of the agile proponents are tiresome ideologues who talk but don’t listen.

So while it is appealing to self-identify as the little boy pointing out the emperor’s lack of apparel, maybe I’m out of touch. Maybe there really is a need for complex modeling initiatives, for up-front requirements documentation. I’m not talking about safety-critical software either — that’s a whole different kettle of fish, and a straw-man for the purposes of this discussion. No, I’m specifically referring to large corporate IT departments, process control software, line automation, etc.

I guess I just don’t know how to experience the waterfall world without being involved in it first-hand. In the agile world, there is hype aplenty — hype sells books and seminars and certifications — but , at the risk of sounding repetitive, experience reports and anecdote just won’t cut it (for either side). It’s extraordinarily difficult in the software community to get at any objective source of information — except surveys, typically.

So who’s up for a PhD establishing the benefits of agile and its ecosystem over waterfall and its suite of tools? Can such a thing even be established?