Forecasting and Calibration

Where the Wisdom of Crowds Runs Out

Aggregated opinion beats most individuals most of the time. The conditions it depends on are specific, and prediction markets sometimes fail to meet them.

A dense cluster of dots with outliers scattered around it

The case for prediction markets rests on aggregation: many people with partial information, each with money at stake, produce a better estimate than most individuals. This is largely true and it is not unconditionally true.

What the result actually requires

Diverse information. Participants must know different things. A crowd where everyone read the same article has one opinion held by many people, which aggregates to that opinion and no more.

Independence. Judgements must be formed separately. Once people can see each other's conclusions, they update toward them, and the aggregate becomes an echo.

Incentives aligned with accuracy. Being right must pay. Markets do this well, which is their main advantage over polls of opinion. Prediction markets versus polls.

Enough participants. Aggregation is a statistical effect. Five people do not produce it.

A mechanism that aggregates rather than averages. A market weights by conviction and capital, which is usually better than a flat average and occasionally worse.

Where prediction markets break each one

Independence, constantly. The price is visible to everyone. A participant who looks at the price before forming a view is not adding independent information; they are anchoring on the aggregate and then being counted as part of it. This is the most serious structural problem and there is no fix within the mechanism.

Diversity, in thin markets. A market with a handful of active participants is a handful of opinions, and its price should not be read as a crowd forecast at all. When a price move means something.

Enough participants, most of the time. The distribution of liquidity is extremely uneven. A small number of markets have real aggregation; most have almost none.

Incentives, at the extremes. When capital cost exceeds the available return, the incentive to correct a price disappears even for people who see it clearly. This is why the tails stay mispriced. Capital lockup and the favourite-longshot bias.

Arrows pointing one way with one pointing away
Independence is the ingredient. It is also the one that fails first.

The specific failure modes

Correlated error. Everyone reading the same forecasters produces a confident consensus that is wrong together. Markets do not protect against this; they aggregate it efficiently.

Attention concentration. Questions with a compelling narrative attract participants; structurally important but dull questions do not.

Capital constraints. A participant who is right and has limited capital cannot move a price as far as a well-capitalised participant who is wrong.

Manipulation in thin markets. Moving a price that nobody is defending is cheap. Academic work suggests manipulation attempts in liquid markets tend to be corrected and can even improve liquidity, but that result depends on there being someone on the other side. In a thin market there is not. Thin market warning signs.

How to use this

Weight a market price by its liquidity. A price with real volume behind it deserves substantial deference. A price with none deserves almost none.

Form your view before looking. The only way to contribute independent information is to generate it independently. Keeping a forecast journal makes this a habit rather than an intention.

Ask what the crowd could not know. Genuine edges usually come from information the participant population does not have, or from a category they systematically misread — not from thinking harder about the same public facts.

Be suspicious of agreeing too easily. If your independent estimate always matches the price, you are probably anchoring.

The balanced view

Liquid prediction markets are among the better forecasting instruments available, and comparing a market price against your own reasoning is a genuinely good habit. The caution is narrower than "markets are wrong": it is that the properties making aggregation work are present in some markets and absent in most, and the price does not announce which kind it is.

Next

The Brier score, and how to measure a forecast properly.

Educational content only. Nothing here is financial advice, a solicitation, or an offer to trade. Prediction markets can move against you and a position can lose its entire value. Where this guide describes a difference between venues it describes a theoretical cross-market edge in observed prices, which is not a profit: fees, spreads, capital lockup, transfer time and differing settlement rules all come out of it first.