Semantic Werks

In a presentation recently, a group of researchers here at UofT showed work they had done on helping the planning committee for ICSE2009 manage sustainability concerns. I have to say I wasn’t impressed, and I speak with a modicum of credibility in the area, having studied it for four five years now. My concern with the sustainability modeling is the same as my critiques of all goal modeling, or, for that matter, any modeling/analysis technique:

• Have you satisfied the stakeholders and added value?
• Would another technique have worked as well?

We always assume that goal modeling is the way to go, but why? Are we so wedded to the technique? To take a problem, satisfy the desire of the ICSE organizers to lower their carbon footprint, and to immediately leap to saying “Goal modeling is the One True Way” smacks of fitting the problem to the solution.

I suggested another technique, such as real options, or the risk-mitigation framework from NASA, might give better value in this situation. What quantitative models give you is a result, an answer to your query. Goal models, particularly i* models, don’t do this. We need some way to query them to give answers. For the models shown, the modeler has to sit with the stakeholders and show them the answers to their queries.

One of the team mentioned that the unique benefits were

1. a way to show evidence for tradeoff decisions;
2. a way to incorporate multiple stakeholders.

They argued for goal modeling as a way to drive analysis and explore options, but that argument sounds like a default response. It was also mentioned that it is hard to assign quantities to all the values. For example, should the organizers hand-out sponsor flyers, or show sponsor logos on monitors?

I agree it isn’t a simple exercise, but isn’t that the point? It seems to me that what the team came up with was essentially a codification of the problem. So to the extent that there is value from better understanding the nature of the ‘wicked’ problem they faced, their work has value. To the extent that they left stakeholders with some answers to that problem, they did nothing of use. There was no scientific method followed. There was no up-front identification of the problem, no consideration of possible solutions, and no identification of how to validate a proposed solution. Absent these components, how are we to understand whether the proposal is useful?