Semantic Werks

The Working Conference on Requirements Engineering (REFSQ) just concluded. It is a great conference with plenty of discussion and provocative ideas.

I tweeted periodically about the conference, and here are some final thoughts:

Statements from the concluding plenary I disagreed with:

• social scientists never do anything with their theories; social science theories are not generalizable; RE should avoid social science techniques.
• You cannot gather data without a theory.
• We shouldn’t wait for data to start creating requirements engineering (RE) theories.
• Replication refers to repeating an experiment, not re-doing a case study.
• Studies shouldn’t just collect data; they should also propose theories.

It was refreshing to be involved in general discussions about the role of theory and empiricism in requirements, as it is something the field has long ignored. Jorge would be happy: there seems to be acknowledgement that we ought to be working towards better theory building in RE. There was also some muted acknowledgement that whatever we did in the past did not work, and that those ‘theories’ — better to call them ‘conjectures’ or just ‘wild guesses’ — need revisiting.

However. Some people don’t seem to understand that there are many ways of doing science in RE. Nearly everyone agrees new techniques are NOT needed; what is necessary is better ways of understanding how existing tools work or don’t work. And social sciences have a lot to teach us here, as a cursory examination of the literature would reveal. This is not physics! And we can’t use “just wing it” as our epistemic theory. Some feel we should jump to wild conjectures about what ought to work, and seek to test that. In fact, what often works better is to adopt grounded theory approaches.

Case in point. Someone mentioned that often in interviews you go to a person and ask about X, and they respond by cursorily mentioning X and then talking about Z five times. Z is the thing you should be interested in! And indeed a grounded theory approach will allow this to appear.

But these are quibbles. I think in general, there is broad acceptance of the need for rigorous empirical techniques, and also acceptance that we need to aim as a community for comprehensive, well-verified explanatory (and perhaps predictive) theories.

I’ll end with a few provocative statements of my own (a theme of the working conference):

I think in requirements it is easy to mistake the trees for the forest. We seem to focus so much on “making RE better” that we lose sight of the ultimate goal, which is to make better (software) products. Every RE theory should tie in to this goal, in my opinion.

And perhaps more controversially, although everyone at the conference probably fears it, it might be the case that all our tools and techniques are irrelevant in the face of the human aspect of the problem. That is, I wager it is easier to remedy poor tools when you have a mature and intelligent organization, that in fact it doesn’t matter what tools you choose. You could do waterfall and be successful (like NASA seems to do).  Here’s a great quote from Watts Humphrey to conclude:

[During my time managing complex projects at IBM] I found that the problems were never technical; they were always management problems.