The California bullet train between San Diego and San Francisco. Lockheed Martin’s Joint Strike Fighter program. Berlin’s Brandenburg Airport. Apple’s AirPower wireless charging pad. These are just a few examples of projects that suffered severe schedule delays and cost overruns, or that were unable to deliver on their promised scope.
Planning projects accurately is notoriously difficult, whether they’re publicly or privately funded, or in domains like construction, technology, pharma, or infrastructure. According to the 2018 “Pulse of the Profession” study conducted by the Project Management Institute, between 2011 to 2018 only about 50% of projects where completed on time and approximately 55% were within budget. Even though firms have been investing in project management techniques since the 1970s, the accuracy of project plans has not improved much.
Inaccurate forecasts involving durations, costs, resources, and benefits are clearly major source of risk for leaders’ careers and organizations’ growth opportunities. For example, firms waste an average of $119 million for every $1 billion spent (11.9%) on projects due to poor project performance. Late or pricey projects can also affect the health of the economy at large. Gross domestic product (GDP) contributions from project-oriented industries are forecasted to reach $20.2 trillion by 2027; major missteps have the potential to chip away at this number.
Forty years ago, psychologist and Nobel prizewinner Daniel Kahneman, along with long-term collaborator Amos Tversky, noted that humans tend to suffer from a planning fallacy: they overpromise and underdeliver by offering unrealistic forecasts of projects’ objectives. Kahneman and Tversky suggested using an outside view to develop more realistic project plans. They proposed using a forecasting technique called reference class forecasting, by which projects’ durations or costs are predicted by comparing a project of interest to a set of past similar projects. Such as outside view is in contrast to the inside view that’s more often taken, where the project is planned with little regard to historical performance and its ability to meet set targets.
Today, changing attitudes toward data collection, data-driven prediction, and decision-making offers unprecedented opportunities in the field of project planning. With data, firms can now operationalize Kahneman and Tversky’s ideas, going beyond their original vision. Using historical data on projects’ initial forecasted completion dates and total costs, in addition to realized or actual expenditures and durations, accuracy estimates can be established. Such estimates can then be used when forecasting and setting new projects’ goals. Here are some examples of places where data-based prediction is occurring, as well as some publicly available resources you can use.
In the U.K., data on project performance has been collected for over a decade now. The HM Treasury’s Green Book provides guidance on how project proposals should be appraised before significant public funds are committed. The appraisal procedure includes an explicit adjustment to account for systematic optimism, sometimes referred to as “optimism bias,” which is the overstatement of benefits and the understatement of durations and costs.
These optimism adjustments use reference class forecasting and are constructed empirically based on historical data deemed relevant. In a study I conducted for the U.K.’s Department for Transport, along with researchers from University College London, Erasmus University Rotterdam, and Warwick Business School, we found that rail infrastructure projects require anywhere from a 64% optimism adjustment (for projects in early definition stages) to a 4% adjustment (for projects that have already completed detailed designs). These adjustments were determined by an analysis of data from thousands of historical projects.
Much of the Green Book’s guidance regarding optimism bias and required adjustments has been inspired by work done by Oxford professor Bent Flyvbjerg. Professor Flyvbjerg has collected data on hundreds of large-scale projects, mostly in the areas of infrastructure, construction, and information technology. Subsets of projects in this dataset serve as relevant reference points when appraising new initiatives.
In the U.S., the Program Management Improvement Accountability Act was signed into law on December 14, 2016. The Act, which aims to improve program and project management practices within the Federal Government, establishes initial guidance for coordinated and government-wide approaches to strengthen project management practices. Aimed at federal agencies, the goal is to improve government performance, including the “use of cost and schedule data to support decision-making.” In the future, this Act will hopefully serve as a significant catalyst to help establish resources for data collection. The fact that the government took this step could also help to establish new norms in project-related data collection.
In both of these examples, the set of projects that are considered in the reference class is identified by human judgement. What if artificial intelligence could help perform this role? Data availability and access to data-driven artificial intelligent algorithms can help take reference class forecasting to the next level, revolutionizing how projects are being planned and how targets are being set.
How? Using detailed project-plan-level data with information on individual tasks, deep learning and artificial intelligence can identify patterns of similarity among project tasks, hierarchies, and precedent relations. nPlan, a London-based startup, is doing this now. The company uses data from tens of thousands of construction projects involving millions of tasks combined with natural language processing techniques to predict project durations and delays. The combination of rich data and proprietary AI capabilities allows nPlan to generate highly accurate and useful forecasts for project completions dates, including information about the risks of delay. Here, AI algorithms learn which patterns are most useful for predicting delays, relaxing the need to declare a reference class upfront.
There have been those who suggest that the availability of data and AI technologies will introduce a “seismic shift” in project planning. Let’s hope that this shift will finally enable us to overcome the planning fallacy, too.
Yael Grushka-Cockayne @ Harvard Business Review
