IT failure statistics
There’s an excellent IT project dashboard from the US government reporting on success/failure rates, project size, and amount of spending (which is frankly jaw-dropping). It is a very useful site, because most of the information we have seems to come from self-interested consultants. It’s certainly in their interests to emphasize how projects are always in a perpetual state of failure. Which doesn’t mean they aren’t, of course. If we look at the very coarse-grained US data, it’s clear that an uncomfortably large number of projects are in trouble. E.g., the figure below shows 7% of projects are in serious trouble, and 34% need attention. 
There’s a reporting bias in the press, too, of course: Man Bites Dog syndrome. If the project works and saves money, it won’t make headlines. For example, the Ontario Telemedicine Network connects physicians, nurses and patients online every single day, with high uptime rates. Every patient that doesn’t have to travel to Toronto from Sudbury saves the government — and the patient — mucho dinero.
There’s another comparison I rarely see, as well: how many projects of any kind are successful? If we look at the recent decision by the Canadian government to dole out ‘stimulus’ money for infrastructure projects (which invariably mean building new roads, for some reason), we see failure rates which are comparable to those in IT. For the 2 year, $4 billion fund, only 25% of projects were finished before the March 2011 deadline, with the most likely scenario being that some 900 projects will not meet the deadline. Manitoba is apparently hoping for favorable weather to meet the target.
Perhaps the difference is that when it is something physical, like a road, it is much harder to leave it half-finished then a piece of software.
I'm a postdoc in software engineering at the University of British Columbia. 
Thanks Neil, this is a very interesting post!
Alberto Bacchelli
2010 August 11 at 03:06
Hi Neil! It’s an interesting subject you bring up which made me wonder if there is any academic studies done recently on the current situation of IT project failure? The sources I see referred to all the time are kind of old.
Thanks. (I actually got here from the mendeley/scrivener post which was also pretty useful)
Patrik Björklund
2011 January 29 at 13:48
I’m not as familiar with the business side of things, but there is good research from Scandinavia on business requirements, software development, and maintenance. For example, Bente Anda’s paper on four companies building the same system.
In general studying these things is challenging because of the complexity. Controlling for all the variables is essentially impossible. So most of the research is case study format, and its poorer cousin, anecdote.
I too find it strange though that so much empirical software research was done in the 80s and then died out. Perhaps the lesson is that the research didn’t generalize very well to other settings (e.g., the Basili studies on NASA are a pretty unique environment).
Neil
2011 January 29 at 23:15
Congrats. Nice post.
julio leite
2011 April 21 at 08:09