55 pages • 1 hour read
Philip E. Tetlock, Dan GardnerSuperforecasting: The Art and Science of Prediction
Philip E. Tetlock, Dan GardnerNonfiction | Book | Adult | Published in 2015A modern alternative to SparkNotes and CliffsNotes, SuperSummary offers high-quality Study Guides with detailed chapter summaries and analysis of major themes, characters, and more.
Chapter 12-AppendixChapter Summaries & Analyses
Chapter 12 Summary: “What’s Next?”
Tetlock reasons that “our expectations of the future are derived from our mental models of how the world works, and every event is an opportunity to learn and improve those models” (251). However, as noted in previous chapters, it’s possible to improve these predictive models only by receiving clear feedback. Forecasting must therefore involve measuring and revising. To improve, forecasters must apply the same evidence-based guidelines used in medical testing.
However, the authors acknowledge that politicians and others in positions of power have a vested interest in keeping forecasting inaccurate. This is because accuracy is far from the only goal of forecasting: Political parties favor polls that make them look good and point to their victories, while campaigners use forecasts to rally their troops, sometimes exaggerating facts and figures in the service of their cause.
Tetlock and Gardner state that the future of forecasting will ultimately depend upon what people demand from it. Tetlock draws attention to early-20th-century doctor Ernest Amory Codman, who demanded better hospital evaluation; many in the medical establishment were as opposed to Codman’s demand as they now are to implementing systems of feedback in forecasting. However, the ultimate success of Codman’s methods inspired evidence-based testing even in nonmedical fields.
The authors argue that to see forecasting improve, people must be willing to think long-term and consider the benefits of score keeping from an historical standpoint. Tetlock and Gardner also state the importance of accepting that forecasting is a work in progress that can always be improved.
The authors assert that while many would like forecasting to answer big questions about the future, what the discipline truly offers is the ability to carefully consider the smaller questions that underlie the big questions. For example, instead of answering whether there will be another Korean war, forecasting is better applied to phenomena that would lead to such a conflict, such as missile launches, nuclear tests, and cyberattacks.
The authors believe that for forecasting to truly thrive, superforecasters must be paired with superquestioners, who ask the “smack-in-the-forehead” questions that seem so obvious and important that we ought to have asked them ourselves (264). Superquestioners’ qualities tend to be more hedgehoglike than foxlike, and they tend to be big-idea people. A partnership of superquestioners and superforecasters would enable tackling big questions without encouraging the destructive polarization that can happen when groups dig in their heels over their differences. The authors reiterate that to improve forecasting, forecasters need to get serious about keeping score on the accuracy of their predictions.
Epilogue Summary
Tetlock and Gardner return to superforecaster Bill Flack and underscore his commitment to accuracy as well as his humility. Bill knows there is no guarantee he will continue to make accurate predictions and is thus committed to acting on feedback and continually improving in the manner of perpetual beta.
Appendix Summary: “Ten Commandments for Aspiring Superforecasters”
This section outlines the 10 guidelines for superforecasting, which are designed to enhance accuracy.
1. Triage: Ensure your questions are at the right level. Aim for a middle ground between questions where rules of thumb offer the answer and those of which the outcomes are so distant that they are truly impossible to predict—for example, the outcome of a general election 13 years into the future.
2. Break seemingly intractable problems into tractable sub-problems: This involves Fermi’s strategy regarding the query about the number of piano tuners in Chicago. When asked a big, complex question, break it into smaller, more accessible questions.
3. Strike the right balance between inside and outside views: Superforecasters are confident that no situation, however rare, is 100% unique. They seek a picture of comparative situations (the outside view) before tackling an individual situation (the inside view).
4. Strike the right balance between under- and overreacting to evidence:
The technique of responding to incoming evidence in just the right amount takes skill and can be cultivated.
5. Look for the clashing causal forces at work in each problem: Consider the counterargument to your initial position. Then, reconcile the two views to get the most nuanced perspective.
6. Strive to distinguish as many degrees of doubt as the problem permits, but no more: Recall that “your uncertainty dial needs more than three settings” (281), which are “definitely,” “maybe,” and “no.” It is far more accurate to come up with numerical odds for predictions.
7. Strike the right balance between under- and overconfidence and between prudence and decisiveness: Straddle the line between circumspection and boldness to defend against the respective forecasting errors of false alarms and misses.
8. Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases: Assess where your judgments went wrong, but avoid overcorrecting your processes when a minor technical error is to blame. Evaluate your successful judgments because errors in reasoning may still be present there.
9. Bring out the best in others and let others bring out the best in you: Teamwork and precision questioning are important.
10. Master the error-balancing bicycle: No amount of theorizing can replace the actual practice of forecasting.
11. Don’t treat commandments as commandments: There is no benefit in always following the rules, but be judicious about breaking them.
Chapter 12-Appendix Analysis
The book’s final sections consider where forecasting will go next, both in history and in the reader’s life. At the beginning of the book, the authors label themselves optimistic skeptics; likewise, by its end, they are cautiously optimistic about forecasting’s development and future impact. On the one hand, more research and resources than ever are available to improve forecasting; on the other hand, many institutions have vested interests in keeping forecasting mediocre and in the service of propaganda rather than accuracy. Such bodies, whether governmental or corporate, will make use of the techniques that lead to poor forecasting, such as preferring vague verbal forecasts to numerical ones and deliberately misinterpreting data to paint a flattering picture of their institution. Such fraudulent forecasting is evident in partisan media that misconstrues statistics to exaggerate a party’s success or potential.
Another obstacle to accurate forecasting is the mass appeal of media forecasters like political commentator Tom Friedman. Here, the authors propose that instead of an either/or, there is a place for both superforecasters and Friedman, who can be dubbed a superquestioner. This pairing would constitute the forecasting dream team of Hedgehogs and Foxes, as hedgehoglike media pundits (superquestioners) can identify the most pressing issues for the future, while superforecasters can go some way toward finding answers. Indeed, the Good Judgment Open website shows the superquestioner and superforecaster mix already in play, as the questions posed incorporate the most controversial and important problems for the near future, such as the meeting of leaders of countries in conflict or the potential banning of controversial media platforms.
The Appendix, which summarizes the skills needed to become a good forecaster, democratizes the mental processes used by superforecasters. Adopting such techniques on a personal level can encourage people to demand higher standards of forecasts delivered in the media. In a media landscape rife with false visions of the present and future, the simple processes of doubting what we know and what we are told can make people more critical media consumers and better truth-hunters.
Unlock all 55 pages of this Study GuidePlus, gain access to 8,800+ more expert-written Study Guides.
Including features:
Featured Collections
Business & Economics
View Collection
Canadian Literature
View Collection
Common Reads: Freshman Year Reading
View Collection
New York Times Best Sellers
View Collection
Politics & Government
View Collection
Psychology
View Collection
Science & Nature
View Collection
Self-Help Books
View Collection
Teams & Gangs
View Collection
The Best of "Best Book" Lists
View Collection