Your Track Record Is a Better Forecaster Than Your Plan

Think of the last kitchen renovation, book manuscript or company reorganization you were part of. Now try to remember the original estimate. If you're like most people, the honest answer is a little embarrassing. The project took longer, sometimes much longer, than anyone said it would.

Psychologists have a name for this: the planning fallacy. It is one of the best-documented habits of the human mind. It also happens to be one of the few mental traps that gets easier to escape the more life you have lived, provided you use that life the right way.

A deadline experiment with real stakes

The idea goes back to Daniel Kahneman and Amos Tversky, who described it in the late 1970s. One of the cleanest demonstrations came in 1994, when psychologists Roger Buehler, Dale Griffin and Michael Ross studied senior students at the University of Waterloo who were writing their honors theses.

The researchers asked each student a simple question: realistically, when will you finish? On average the students said about 34 days. The actual average was about 55 days. Fewer than a third finished by the date they had called their most realistic guess.

Then came the telling part. The researchers had also asked for a worst-case estimate: how long would it take if everything went as poorly as it possibly could? The average worst case was about 49 days. Fewer than half finished even by that date.

These were bright young people predicting their own work, on a project they understood well. They still missed by weeks.

Why we keep doing this

Across five studies, Buehler and his colleagues looked for the cause. When people predicted their own completion times, they mostly thought about the future. They pictured the plan: draft the chapters, revise, print, submit. What they rarely did, unprompted, was think about how long similar projects had taken them before.

When they did remember past delays, they tended to explain them away. That was the semester I was sick. That was the year the printer broke. Each delay looked like a one-off, so none of them seemed to count.

Two other findings are worth knowing. When people predicted how long someone else would take, they were more pessimistic and leaned more heavily on that person's track record. And when participants were explicitly asked to connect their past experiences to the new prediction, the optimistic bias disappeared in that study.

In other words, the information needed to forecast well was already in people's heads. They just weren't consulting it.

Even the expert fell for it

Kahneman told a story on himself that makes the point better than any experiment. Decades ago, he was part of a team at the Hebrew University writing a textbook and curriculum on judgment and decision making for Israeli high schools. One day he asked everyone, independently, to estimate when they would hand a finished book to the Ministry of Education. The answers clustered between a year and a half and two and a half years.

Then he asked the curriculum expert on the team a different question: how long had other teams like this one taken? The expert thought for a moment and admitted that about 40 percent of such teams never finished, and that he couldn't think of one that had finished in less than about seven years.

Same person, two completely different forecasts. The first came from looking at the plan. The second came from looking at the record. Kahneman later wrote that the team pressed on anyway, and the book took about eight years. He called the second way of thinking the "outside view." Planners today call it reference class forecasting.

Where experience becomes an advantage

Here is the part that matters for anyone with a few decades behind them. A 25-year-old planning a home renovation has very little personal history to consult. You have a lot. You have watched projects run long, budgets stretch, hires take longer to find than expected, and "two quick meetings" turn into a quarter.

That history is data. It is also, as the research shows, data we routinely ignore in favor of a tidy plan. The planning fallacy doesn't disappear with age. But the cure for it, an honest look at your own track record, grows more powerful every year, as long as you actually use it.

That makes this a skill worth practicing on purpose, whether you're estimating a consulting engagement, a move to a new city, a book, or the timeline for handing a business to the next generation.

Try this: the two-question forecast

The next time you need to estimate how long something will take, or what it will cost, ask two questions before you commit to a number.

• The plan question: If things go roughly as I expect, how long will this take? Write the answer down. This is your inside view.
• The record question: The last three times I, or people I know, did something like this, how long did it actually take compared with the plan? Write those numbers down too. This is your outside view.

If the second number is bigger than the first, and it usually is, take it seriously. Start from the record, then adjust only for differences you can name specifically. "This time I'll be more disciplined" doesn't count. "This time the contractor is already booked and the permits are approved" might.

Two habits make this easier over time:

• Keep a simple estimate log. For any project longer than a few weeks, note your predicted finish date and the actual one. After a year you'll have a personal reference class that's worth more than any productivity app.
• Ask an outsider. Because we're more realistic about other people's projects, describe yours to a trusted friend and ask how long they think it will take. Their guess will often be closer than yours.

The point isn't to become pessimistic. It's to let your past do some of the forecasting. You earned that experience; you might as well let it plan for you.

Sources

• Buehler, R., Griffin, D., & Ross, M. (1994). Exploring the "planning fallacy": Why people underestimate their task completion times. Journal of Personality and Social Psychology, 67(3), 366–381. APA PsycNet record · Full text (PDF)
• Kahneman, D., & Tversky, A. (1979). Intuitive prediction: Biases and corrective procedures. TIMS Studies in Management Science, 12, 313–327.
• Kahneman, D., & Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking. Management Science, 39(1), 17–31. PDF
• Kahneman's account of the curriculum project, interview with Joseph Walker (2023), excerpted at Conversable Economist. He also tells the story in Thinking, Fast and Slow (2011), chapter 23.