zlacker

Your story reminded me of Bent Flyvbjerg and Dan Gardner's book "How Big Things Get Done". It's a pop science book communicating the research of Flyvbjerg & collaborators who collected data of actual schedule & budget outcomes for many large projects, along with recording the estimated schedule and budgets from the time the go/nogo decision was made to invest in the projects.

For accurate schedule and budget estimates, Flyvbjerg strongly recommends figuring out which broader class of projects your project belongs to, then going and looking at accurate historical data with actual schedule and budget outcomes for projects in that class, and using historical averages as your estimates.

There's a great table in an appendix at the back of the book, offering statistics for each category of project, ranking them by mean cost overrun.

The absolute worst category of project, for mean cost overruns, is nuclear storage, with a mean cost overrun of 238%.

IT projects are the 5th worst category of project, with a mean cost overrun of 73%, behind nuclear storage, olympic games, nuclear power and hydroelectric dams.

The table also has statistics on "what percentage of projects has a cost overrun of 50% or greater" and "of those projects with a cost overrun of 50% or greater, what is their mean cost overrun". For nuclear storage projects, 48% of them have a cost overrun of 50% or greater, and of those, the mean cost overrun is 427% (!).

For IT projects, 18% of them have a cost overrun of 50% or greater, and of those, the mean cost overrun is 447% (!!).

Some of the chapters in the book discuss some of the structural or political pressures that set projects up to fail --- e.g. in some fields its an open secret that estimates are always wildly optimistic, as if the estimates were actually realistic, no one would ever agree to fund a project.