Semantic Werks

This post initially was going to be commenting on a recent report on requirements and business analysis. However, along the way I came to read up more on the ‘CHAOS’ reports, and a fascinating discussion about their methodology (and business model), which I think ties in directly with the latest report.

The new report is from IAG, a company that sells requirements consulting services. They released a technical whitepaper on requirements use in business. Titled “The Impact of Business Requirements on the Success of Technology Projects”, the report concludes that 68% of companies will have their projects fail due to bad requirements practices.

This is a company with a vested interest in these results (it’s akin to Merck reporting on a new painkiller). As with most of these reports, such as the Standish CHAOS report, there is no methodology writeup, the participants are unidentified, and there is no assessment of validity and reliability.

They mention a survey of over 100 enterprises, with business application projects costing(?) over $250k. The survey instrument is not available publicly. As a result, the report is replete with statistics and numbers of unknown origin. For example, “over 70% of companies in the upper third of requirements discovery capability reported having a successful project”. What is the upper third of ‘requirements discovery capability’? How is membership assessed? What is a successful project? The rest of the report, it seems to me, can be summed up thusly: “companies that use our Requirements Discovery Process™deliver better projects on-time and under-budget”. Hardly surprising.

See also:

• A report on the reliability of the oft-quoted Standish survey.
• An attempt to extract methodology from Standish and their rebuttal to methodological questioning (which seems to amount to “Tough”).
• A well-known commentator on the usefulness of IT Consultants.