Forecasting and Calibration
Base rates, scoring rules, calibration curves, and telling a good forecast from a lucky one.
Forecasting and Calibration · 3 min readStart With the Base Rate, AlwaysThe single highest-return forecasting habit is asking how often this kind of thing usually happens before thinking about this particular case.
Forecasting and Calibration · 3 min readCalibration Curves and How to Read Your OwnA calibrated forecaster is right 70% of the time when they say 70%. Building the curve, reading its shape, and what each kind of deviation tells you.
Forecasting and Calibration · 3 min readKeeping a Forecast Journal That Is Actually UsefulMost forecasting improvement comes from records, not from cleverness. What to write down, what to leave out, and how to review it without flattering yourself.
Forecasting and Calibration · 3 min readWhere the Wisdom of Crowds Runs OutAggregated opinion beats most individuals most of the time. The conditions it depends on are specific, and prediction markets sometimes fail to meet them.
Forecasting and Calibration · 3 min readThe Brier Score, Explained ProperlyThe standard way to score a probabilistic forecast, what a given number actually means, and why it rewards honesty rather than confidence.
Forecasting and Calibration · 3 min readLog Scoring, and Why Confident Errors Cost So MuchThe other proper scoring rule. It is unbounded, it punishes confident mistakes severely, and that severity is the argument for using it.
Forecasting and Calibration · 3 min readUpdating on News Without OverreactingNew information should move your forecast by an amount proportional to how surprising it is. Most people move too far on noise and not far enough on signal.
Forecasting and Calibration · 3 min readThe Habits That Actually Correlate With AccuracyResearch on forecasting tournaments found consistent behavioural differences in the best performers. Most of them are habits rather than talents.
Forecasting and Calibration · 3 min readLuck, Skill, and How Many Forecasts You Actually NeedTwenty correct calls prove very little. A short look at how much evidence it takes to distinguish a real edge from an ordinary run of good luck.
Forecasting and Calibration · 3 min readCombining Several Forecasts Into a Better OneAveraging estimates usually beats picking the best one. Which average to use, why extremising helps, and how a market price is itself an aggregate.