The Habits That Actually Correlate With Accuracy

Large forecasting tournaments have produced a reasonably consistent picture of what separates the best forecasters from everyone else. The encouraging finding is how much of it is method rather than ability.

Start outside, then move inside

Begin with the base rate for the reference class, then adjust for specifics. The reverse order — build a story, then sanity-check it — produces worse forecasts and more confident ones. Base rates first.

Break the question apart

Decompose into parts you can estimate separately. For "will this be completed by March", ask: will it start on time, how long does this kind of thing take, what is the chance of a specific known blocker.

Estimating four tractable things and combining them beats estimating one intractable thing directly. It also makes the reasoning inspectable later, which is where most of the learning happens.

Use fine gradations

The best forecasters distinguish 63% from 68%. This sounds like false precision and empirically is not: forecasters who use more granular numbers score better, and rounding everything to the nearest ten per cent measurably degrades accuracy.

The market equivalent is that a one-cent tick carries real information in a liquid market. When a price move means something.

Update often and in small steps

Frequent small revisions beat rare large ones. This requires actually revisiting your forecasts, which is the part most people skip. A forecast made in January and never touched is a January forecast being scored in June.

Large sudden updates are sometimes right, but a pattern of them usually indicates that the earlier numbers were not being maintained. Updating on news.

Actively look for the opposite case

Not as a ritual. Specifically: what is the strongest version of the argument that I am wrong, and what would I expect to see if it were right.

Then check whether you are seeing it. This is the single habit that most reliably catches a forecast that has drifted into being a position.

Separate the forecast from the stake

Once you hold a position, your forecasts on that question become friendlier to it. This is automatic and it is invisible from the inside. The only defence is a written record made before you took the position. Behavioural traps.

Keep score

Every consistent finding here comes from people whose forecasts were recorded and scored. Without records you have a feeling about how you are doing, and the feeling is wrong in a predictable direction. Keeping a forecast journal, the Brier score.

Treat beliefs as revisable

The temperamental finding: the best forecasters were comfortable changing their minds and did not treat a previous forecast as a commitment. Being wrong was information rather than an embarrassment.

This is harder to adopt deliberately than the mechanical habits, but it is the one that makes the others work.

What did not predict accuracy

Notably: subject-matter credentials, strong ideological commitment, and confidence. Domain knowledge helped somewhat. Method helped more.

Applying this to markets

Form your estimate before looking at the price. Compare. Where you disagree materially, write down why, then check later whether your reasons were any good.

After a few dozen of those you will know whether your disagreements with the market carry information. That is worth more than any strategy, because it tells you whether you should have a strategy at all.

Next

Luck versus skill in small samples — how many forecasts before you know anything.