80 Top Quotes From Superforecasting
Superforecasting: The Art and Science of Prediction by Philip E. Tetlock and Dan Gardner explores the intriguing world of forecasting and prediction, delving into the methods and characteristics of superforecasters – individuals with an exceptional ability to make accurate and insightful predictions about the future. Based on extensive research and analysis, the book challenges the notion of unattainable expertise in forecasting and demonstrates that ordinary people, with the right mindset and techniques, can outperform experts and traditional prediction models. Tetlock and Gardner introduce the concept of "foxes" and "hedgehogs," where foxes, characterized by open-mindedness, pragmatism, and adaptability, tend to be more accurate forecasters than rigid and overconfident hedgehogs.
The book explores the qualities and habits that set superforecasters apart, such as actively seeking feedback, updating beliefs based on new evidence, and engaging in collaborative thinking. It also examines the role of probabilistic thinking in making more nuanced and realistic predictions. "Superforecasting" provides valuable lessons for decision-makers, policymakers, and anyone seeking to improve their ability to anticipate future events accurately. As the world becomes increasingly complex and uncertain, this book equips readers with the tools to enhance their forecasting skills and become more adept at making informed, data-driven predictions. (Superforecasting Summary).
Superforecasting Quotes
"For superforecasters, beliefs are hypotheses to be tested, not treasures to be guarded.” (Meaning)
"For scientists, not knowing is exciting. It’s an opportunity to discover; the more that is unknown, the greater the opportunity.”
"If you don’t get this elementary, but mildly unnatural, mathematics of elementary probability into your repertoire, then you go through a long life like a one-legged man in an ass-kicking contest.”
"Churchill sent Keynes a cable reading, ‘Am coming around to your point of view.’ His Lordship replied, ‘Sorry to hear it. Have started to change my mind.’ ”7”
"The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function,” F. Scott Fitzgerald”
"Consensus is not always good; disagreement not always bad. If you do happen to agree, don’t take that agreement—in itself—as proof that you are right. Never stop doubting.”
"If you have to plan for a future beyond the forecasting horizon, plan for surprise. That means, as Danzig advises, planning for adaptability and resilience.”
"It’s very hard to master and if you’re not learning all the time, you will fail. That being said, humility in the face of the game is extremely different than humility in the face of your opponents.”
"It is wise to take admissions of uncertainty seriously,” Daniel Kahneman noted, “but declarations of high confidence mainly tell you that an individual has constructed a coherent story in his mind, not necessarily that the story is true.”
"It follows that the goal of forecasting is not to see what’s coming. It is to advance the interests of the forecaster and the forecaster’s tribe.”
"There is no divinely mandated link between morality and competence.”
"It was the absence of doubt—and scientific rigor—that made medicine unscientific and caused it to stagnate for so long.”
"Fuzzy thinking can never be proven wrong. And only when we are proven wrong so clearly that we can no longer deny it to ourselves will we adjust our mental models of the world—producing a clearer picture of reality. Forecast, measure, revise: it is the surest path to seeing better.”
"Be careful about making assumptions of expertise, ask experts if you can find them, reexamine your assumptions from time to time.”
"All models are wrong,” the statistician George Box observed, “but some are useful.”
"Foresight isn’t a mysterious gift bestowed at birth. It is the product of particular ways of thinking, of gathering information, of updating beliefs. These habits of thought can be learned and cultivated by any intelligent, thoughtful, determined person.”
"Knowing what we don’t know is better than thinking we know what we don’t.”
"Here’s a very simple example,” says Annie Duke, an elite professional poker player, winner of the World Series of Poker, and a former PhD-level student of psychology. “Everyone who plays poker knows you can either fold, call, or raise [a bet]. So what will happen is that when a player who isn’t an expert sees another player raise, they automatically assume that that player is strong, as if the size of the bet is somehow correlated at one with the strength of the other person’s hand.” This is a mistake.”
"And yet this stagnation is a big reason why I am an optimistic skeptic. We know that in so much of what people want to predict—politics, economics, finance, business, technology, daily life—predictability exists, to some degree, in some circumstances. But there is so much else we do not know.”
"The facts change, I change my mind.”
"They aren’t gurus or oracles with the power to peer decades into the future, but they do have a real, measurable skill at judging how high-stakes events are likely to unfold three months, six months, a year, or a year and a half in advance. The other conclusion is what makes these superforecasters so good. It’s not really who they are. It is what they do. Foresight isn’t a mysterious gift bestowed at birth. It is the product of particular ways of thinking, of gathering information, of updating beliefs.”
"All who drink of this treatment recover in a short time, except those whom it does not help, who all die,” he wrote. “It is obvious, therefore, that it fails only in incurable cases.”
"The fundamental message: think. If necessary, discuss your orders. Even criticize them. And if you absolutely must—and you better have a good reason—disobey them.”
"The one undeniable talent that talking heads have is their skill at telling a compelling story with conviction, and that is enough.”
"success can lead to acclaim that can undermine the habits of mind that produced the success.”
"The ultimate goal of science is uncertainty’s total eradication.”
"It is wise to take admissions of uncertainty seriously,” Daniel Kahneman noted, “but declarations of high confidence mainly tell you that an individual has constructed a coherent story in his mind, not necessarily that the story is true.”17”
"In one of history’s great ironies, scientists today know vastly more than their colleagues a century ago, and possess vastly more data-crunching power, but they are much less confident in the prospects for perfect predictability.”
"Statisticians sleeping with their feet in an oven and their head in a freezer because the average temperature is comfortable.”
"I have been struck by how important measurement is to improving the human condition,” Bill Gates wrote. “You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal….This may seem basic, but it is amazing how often it is not done and how hard it is to get right.”
"In describing how we think and decide, modern psychologists often deploy a dual-system model that partitions our mental universe into two domains. System 2 is the familiar realm of conscious thought. It consists of everything we choose to focus on. By contrast, System 1 is largely a stranger to us. It is the realm of automatic perceptual and cognitive operations—like those you are running right now to transform the print on this page into a meaningful sentence or to hold the book while reaching for a glass and taking a sip. We have no awareness of these rapid-fire processes but we could not function without them. We would shut down.”
"Forget the old advice to think twice. Superforecasters often think thrice—and sometimes they are just warming up to do a deeper-dive analysis.”
"Like everyone else, scientists have intuitions. Indeed, hunches and flashes of insight—the sense that something is true even if you can’t prove it—have been behind countless breakthroughs. The interplay between System 1 and System 2 can be subtle and creative. But scientists are trained to be cautious. They know that no matter how tempting it is to anoint a pet hypothesis as The Truth, alternative explanations must get a hearing. And they must seriously consider the possibility that their initial hunch is wrong. In fact, in science, the best evidence that a hypothesis is true is often an experiment designed to prove the hypothesis is false, but which fails to do so. Scientists must be able to answer the question “What would convince me I am wrong?” If they can’t, it’s a sign they have grown too attached to their beliefs.”
"Scientists must be able to answer the question “What would convince me I am wrong?” If they can’t, it’s a sign they have grown too attached to their beliefs.”
"We are all forecasters. When we think about changing jobs, getting married, buying a home, making an investment, launching a product, or retiring, we decide based on how we expect the future will unfold.”
"The foundations of our decision making were gravely flawed,” McNamara wrote in his autobiography. “We failed to analyze our assumptions critically, then or later."”
"But when big events happen—markets crash, wars loom, leaders tremble—we turn to the experts, those in the know. We look to people like Tom Friedman.”
"Forecasters who can’t cope with the dissonance risk making the most serious possible forecasting error in a conflict: underestimating your opponent.”
"Forecasters who see illusory correlations and assume that moral and cognitive weakness run together will fail when we need them most.”
"With firmness in the right, as God gives us to see the right, let us strive on to finish the work we are in,” Abraham Lincoln”
"There are no certainties in life—not even death and taxes if we assign a nonzero probability to the invention of technologies that let us upload the contents of our brains into a cloud-computing network and the emergence of a future society so public-spirited and prosperous that the state can be funded with charitable donations.”
"We don’t want intelligence analysts to assume jihadist groups must be inept or that vicious regimes can’t be creatively vicious.”
"The difference between heavyweights and amateurs, she said, is that the heavyweights know the difference between a 60⁄40 bet and a 40⁄60 bet.”
"Once we know the outcome of something, that knowledge skews our perception of what we thought before we knew the outcome: that’s hindsight bias. Baruch Fischhoff was the first to document the phenomenon in a set of elegant experiments.”
"It is natural to identify our thinking with the ideas, images, plans, and feelings that flow through consciousness. What else could it be? If I ask, “Why did you buy that car?” you can trot out reasons: “Good mileage. Cute style. Great price.” But you can only share thoughts by introspecting; that is, by turning your attention inward and examining the contents of your mind. And introspection can only capture a tiny fraction of the complex processes whirling inside your head—and behind your decisions.”
"More often forecasts are made and then…nothing. Accuracy is seldom determined after the fact and is almost never done with sufficient regularity and rigor that conclusions can be drawn. The reason? Mostly it’s a demand-side problem: The consumers of forecasting—governments, business, and the public—don’t demand evidence of accuracy. So there is no measurement. Which means no revision. And without revision, there can be no improvement.”
"Consumers of forecasting will stop being gulled by pundits with good stories and start asking pundits how their past predictions fared—and reject answers that consist of nothing but anecdotes and credentials.”
"It’s a rare day when a journalist says, “The market rose today for any one of a hundred different reasons, or a mix of them, so no one knows.”
"Archilochus: “The fox knows many things but the hedgehog knows one big thing.”
"When the scientist tells you he does not know the answer, he is an ignorant man. When he tells you he has a hunch about how it is going to work, he is uncertain about it. When he is pretty sure of how it is going to work, and he tells you, “This is the way it’s going to work, I’ll bet,” he still is in some doubt. And it is of paramount importance, in order to make progress, that we recognize this ignorance and this doubt. Because we have the doubt, we then propose looking in new directions for new ideas. The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test.11”
"If you have the time to think before making a big decision, do so—and be prepared to accept that what seems obviously true now may turn out to be false later.”
"Take the price of oil, long a graveyard topic for forecasting reputations.”
"But declarations of high confidence mainly tell you that an individual has constructed a coherent story in his mind, not necessarily that the story is true.”
"And only when we are proven wrong so clearly that we can no longer deny it to ourselves will we adjust our mental models of the world—producing a clearer picture of reality. Forecast,”
"In the Paleolithic world in which our brains evolved, that’s not a bad way of making decisions. Gathering all evidence and mulling it over may be the best way to produce accurate answers, but a hunter-gatherer who consults statistics on lions before deciding whether to worry about the shadow moving in the grass isn’t likely to live long enough to bequeath his accuracy-maximizing genes to the next generation. Snap judgments are sometimes essential. As Daniel Kahneman puts it, “System 1 is designed to jump to conclusions from little evidence.””
"When the facts change, I change my mind,” the legendary British economist John Maynard Keynes declared.”
"Not knowing is exciting. It's an opportunity to discover.”
"When you combine the judgments of a large group of people to calculate the “wisdom of the crowd” you collect all the relevant information that is dispersed among all those people. But none of those people has access to all that information. One person knows only some of it, another knows some more, and so on.”
"This approach, built on the “wisdom of the crowd” concept, has been called “the crowd within.” The billionaire financier George Soros exemplifies it. A key part of his success, he has often said, is his mental habit of stepping back from himself so he can judge his own thinking and offer a different perspective—to himself.”
"Like everyone else, scientists have intuitions. Indeed, hunches and flashes of insight—the sense that something is true even if you can’t prove it—have been behind countless breakthroughs. The interplay between System 1 and System 2 can be subtle and creative.”
"The truth is, the truth is elusive.”
"I’d rather be a bookie than a goddamn poet,” was his legendary response.”
"To illustrate, imagine”
"When people gather and discuss in a group, independence of thought and expression can be lost. Maybe one person is a loudmouth who dominates the discussion, or a bully, or a superficially impressive talker, or someone with credentials that cow others into line. In so many ways, a group can get people to abandon independent judgment and buy into errors. When that happens, the mistakes will pile up, not cancel out. This is the root of collective folly, whether it’s Dutch investors in the seventeenth century, who became collectively convinced that a tulip bulb was worth more than a laborer’s annual salary, or American home buyers in 2005, talking themselves into believing that real estate prices could only go up. But loss of independence isn’t inevitable in a group, as JFK’s team showed during the Cuban missile crisis. If forecasters can keep questioning themselves and their teammates, and welcome vigorous debate, the group can become more than the sum of its parts.”
"Knowledge is something we can all increase, but only slowly. People who haven’t stayed mentally active have little hope of catching up to lifelong learners.”
"There’s also the “premortem,” in which the team is told to assume a course of action has failed and to explain why—which makes team members feel safe to express doubts they may have about the leader’s plan. But the super-teams did not start with leaders and norms, which created other challenges.”
"Stock prices do not always reflect the true value of companies, so an investor should study a company thoroughly and really understand its business, capital, and management when deciding whether it had sufficient underlying value to make an investment for the long term worthwhile.”
"The CIA was training Cuban exiles to land in Cuba and launch a guerrilla war against the new government of Fidel Castro.”
"Foresight isn’t a mysterious gift bestowed at birth. It is the product of particular ways of thinking, of gathering information, of updating beliefs.”
"For centuries, it hobbled progress in medicine. When physicians finally accepted that their experience and perceptions were not reliable means of determining whether a treatment works, they turned to scientific testing—and medicine finally started to make rapid advances. The same revolution needs to happen in forecasting.”
"If forecasters can keep questioning themselves and their teammates, and welcome vigorous debate, the group can become more than the sum of its parts.”
"But groups also let people share information and perspectives. That’s good. It helps make dragonfly eye work, and aggregation is critical to accuracy.”
"On average, teams were 23% more accurate than individuals.”
"People take things differently. What one person would consider a helpful inquiry another might take as an aggressive criticism.”
"I have been struck by how important measurement is to improving the human condition,” Bill Gates wrote. “You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal….”
"The one undeniable talent that talking heads have is their skill at telling a compelling story with conviction, and that is enough. Many have become wealthy peddling forecasting of untested value to corporate executives, government officials, and ordinary people who would never think of swallowing medicine of unknown efficacy and safety but who routinely pay for forecasts that are as dubious as elixirs sold from the back of a wagon.”
"Natural as such thinking may be, it is problematic. Lay out the tangled chain of reasoning in a straight line and you see this: “The probability that I would meet the love of my life was tiny. But it happened. So it was meant to be. Therefore the probability that it would happen was 100%.” This is beyond dubious. It’s incoherent. Logic and psycho-logic are in tension.”
"Was one of the worst—arguably the worst—intelligence failure in modern history.”
"Research on calibration—how closely your confidence matches your accuracy—routinely finds people are too confident.10 But overconfidence is not an immutable law of human nature. Meteorologists generally do not suffer from it. Neither do seasoned bridge players. That’s because both get clear, prompt feedback.”
"Predictably, psychologists who test police officers’ ability to spot lies in a controlled setting find a big gap between their confidence and their skill. And that gap grows as officers become more experienced and they assume, not unreasonably, that their experience has made them better lie detectors. As a result, officers grow confident faster than they grow accurate, meaning they grow increasingly overconfident.”
"Once we know the outcome of something, that knowledge skews our perception of what we thought before we knew the outcome: that’s hindsight bias.”
"In Germany’s war academies, scenarios were laid out and students were invited to suggest solutions and discuss them collectively. Disagreement was not only permitted, it was expected, and even the instructor’s views could be challenged because he “understood himself to be a comrade among others,”
"A forecaster who doesn’t adjust her views in light of new information won’t capture the value of that information, while a forecaster who is so impressed by the new information that he bases his forecast entirely on it will lose the value of the old information that underpinned his prior forecast. But the forecaster who carefully balances old and new captures the value in both—and puts it into her new forecast. The best way to do that is by updating often but bit by bit.”
"Scientist and science fiction writer Arthur C. Clarke famously observed, “Any sufficiently advanced technology is indistinguishable from magic.”
"what is often forgotten is that the Nazis did not create the Wehrmacht. They inherited it. And it could not have been more different from the unthinking machine we imagine—as the spectacular attack on the Belgian fortress of Eben Emael demonstrated.”
"So finding meaning in events is positively correlated with well-being but negatively correlated with foresight. That sets up a depressing possibility: Is misery the price of accuracy?”
"Amount of luck in tournament determines amount of regression to the mean from one year to the next.”
"Then came the waiting, a test of patience for even the tenured.”
"How skillfully leaders perform this balancing act determines how successfully their organizations can cultivate superteams that can replicate the balancing act down the chain of command. And this is not something that one isolated leader can do on his own. It requires a wider willingness to hear unwelcome words from others—and the creation of a culture in which people feel comfortable speaking such words.”
"Snap judgments are sometimes essential. As Daniel Kahneman puts it, “System 1 is designed to jump to conclusions from little evidence.”13”
"Kudlow was optimistic. “There is no recession,” he wrote. “In fact, we are about to enter the seventh consecutive year of the Bush boom.”
"Doubt is not a fearful thing,” Feynman observed, “but a thing of very great value.”10 It’s what propels science forward. When the scientist tells you he does not know the answer, he is an ignorant man. When he tells you he has a hunch about how it is going to work, he is uncertain about it. When he is pretty sure of how it is going to work, and he tells you, “This is the way it’s going to work, I’ll bet,” he still is in some doubt. And it is of paramount importance, in order to make progress, that we recognize this ignorance and this doubt. Because we have the doubt, we then propose looking in new directions for new ideas. The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test.”
"You see the shadow. Snap! You are frightened—and running. That’s the “availability heuristic,” one of many System 1 operations—or heuristics—discovered by Daniel Kahneman, his collaborator Amos Tversky, and other researchers in the fast-growing science of judgment and choice.”
"Imagine you suffer from insomnia and haven’t slept properly in days and you lose your temper and shout at a colleague. Then you apologize. What does this incident say about you? It says you need your sleep. Beyond that, it says nothing. But imagine you see someone who snaps, shouts, then apologizes and explains that he has insomnia and hasn’t slept properly in days. What does that incident say about that person? Logically, it should say about him what it said about you, but decades of research suggest that’s not the lesson you will draw. You will think this person is a jerk. Psychologists call this the fundamental attribution error. We are fully aware that situational factors—like insomnia—can influence our own behavior, and we rightly attribute our behavior to those factors, but we routinely don’t make the same allowance for others and instead assume that their behavior reflects who they are. Why did that guy act like a jerk? Because he is a jerk. This is a potent bias. If a student is told to speak in support of a Republican candidate, an observer will tend to see the student as pro-Republican even if the student only did what she was told to do—and even if the observer is the one who gave the order! Stepping outside ourselves and seeing things as others do is that hard.”
― Quotes from the book Superforecasting by Dan Gardner
Superforecasting Author
Dan Gardner is an acclaimed journalist, author, and risk expert known for his work in exploring human perception and decision-making in the face of uncertainty. Through his best-selling book "Risk: The Science and Politics of Fear," co-authored with Philip Tetlock, Gardner delves into the psychological factors that influence our perception of risk and how societal perceptions of danger often deviate from actual risk assessments. He challenges the biases and cognitive shortcuts that can lead individuals and policymakers to overreact to certain risks while neglecting others that may be more significant. Gardner's insightful analysis and evidence-based approach to risk have shed light on the complex interplay between media, politics, and public opinion. By encouraging a more nuanced and rational understanding of risk, Dan Gardner has inspired readers to make more informed decisions and cultivate a healthier relationship with uncertainty in an increasingly interconnected world.
Chief Editor
Tal Gur is an author, founder, and impact-driven entrepreneur at heart. After trading his daily grind for a life of his own daring design, he spent a decade pursuing 100 major life goals around the globe. His journey and most recent book, The Art of Fully Living, has led him to found Elevate Society.
