Keeping a Forecast Journal That Is Actually Useful
Most forecasting improvement comes from records, not from cleverness. What to write down, what to leave out, and how to review it without flattering yourself.
Human memory edits forecasts after the fact. You remember having been less sure than you were about things that went wrong, and more sure about things that went right. This is automatic, universal, and it makes unaided self-assessment worthless.
A journal fixes it. Not a complicated one.
What a good entry contains
The question, stated precisely enough to settle. If you cannot say what would make it true, you cannot score it. This constraint alone improves thinking.
The date.
Your probability. A number, not a word. "Likely" cannot be scored.
The base rate you started from, and the reference class. So you can later find out whether your reference classes are any good, separately from whether your adjustments are. Base rates first.
The market price, if there is one. The most valuable single field. It lets you ask whether your deviations from the market carry information.
Two or three lines of reasoning. Specifically: what would change your mind. This is the field that pays off most at review time.
What to leave out
Long essays. If entries take fifteen minutes you will stop making them within a fortnight. Three minutes is the target.
Confidence adjectives. Numbers only.
Post-hoc edits. If your view changes, make a new entry with a new date. The old one stays. The sequence of your views over time is the interesting object, and editing destroys it.
The review
Every few months, not every week.
Score the resolved ones. Brier or log, consistently. The Brier score.
Plot calibration. Once you have enough. Calibration curves.
Compare against the market. Take every forecast where you disagreed with the price by more than a few cents. How did those resolve? If the answer is "about as the market implied", your disagreements are noise, and that is genuinely valuable to know.
Read your own reasoning on the ones you got badly wrong. Not to feel bad — to spot patterns. Most people find two or three recurring failure modes and those are worth more than any amount of new technique.
The patterns people find
Overconfidence at the extremes. Nearly everyone. The 95% forecasts come in around 85%.
Recency. Forecasts made right after a dramatic event lean toward that event repeating.
Attachment to a position. Forecasts on questions where you hold a trade are systematically friendlier to your trade. This one is worth checking for explicitly, because it is invisible from the inside. Behavioural traps.
Anchoring on the market. If your numbers are always within three cents of the price, you are reading, not forecasting. Useful to know before concluding you have an edge.
Why this beats reading more
Forecasting skill is trainable and the training signal is feedback. Without records there is no feedback, only a story about how you have been doing.
Six months of a boring journal does more for accuracy than any amount of additional reading, including this page.
Next
The limits of crowd wisdom — when the market is the thing that is wrong.
Educational content only. Nothing here is financial advice, a solicitation, or an offer to trade. Prediction markets can move against you and a position can lose its entire value. Where this guide describes a difference between venues it describes a theoretical cross-market edge in observed prices, which is not a profit: fees, spreads, capital lockup, transfer time and differing settlement rules all come out of it first.
Keep reading
Forecasting and Calibration · 3 min readCalibration Curves and How to Read Your OwnA calibrated forecaster is right 70% of the time when they say 70%. Building the curve, reading its shape, and what each kind of deviation tells you.
Forecasting and Calibration · 3 min readWhere the Wisdom of Crowds Runs OutAggregated opinion beats most individuals most of the time. The conditions it depends on are specific, and prediction markets sometimes fail to meet them.
Forecasting and Calibration · 3 min readStart With the Base Rate, AlwaysThe single highest-return forecasting habit is asking how often this kind of thing usually happens before thinking about this particular case.
Kalshi · 3 min readWhat CFTC Regulation Means for an Event ExchangeA designated contract market licence changes what can be listed, how funds are held and who can participate. A plain explanation of the parts that affect users.