Working With Public Prediction Market APIs
What public endpoints give you across venues, the fields that matter, and the habits that keep a data pipeline working and welcome.
Prediction market data is unusually pleasant to work with: small, well structured, and about questions a human can read. This is a good domain to learn data engineering in.
Everything here concerns public, read-only data. Nothing in it involves an account, and no legitimate price-viewing tool needs credentials, keys or seed phrases of any kind.
The fields that matter
Identity. A stable identifier for the market, plus the human-readable question.
Resolution criteria, in full. Store the whole text. It is the actual contract, it changes rarely, and you will need it later to explain why two venues resolved differently. Almost everyone stores the title and regrets it.
Status and close time. Open, closed, settled. With timestamps.
Best bid and best offer, with sizes. The live price. Not the last trade.
Depth, where available. Several levels each side. The field most people skip and most need later. Reading an order book.
Last trade, with its own timestamp. Useful as a staleness indicator, not as a price.
Volume over a recent window. Not cumulative since listing, which is close to useless as a liquidity measure. Volume versus open interest.
The venue's own observation timestamp. The single most important field in the whole record.
Why the venue timestamp matters so much
Every cross-venue comparison is only as good as the older of its two observations. Without the venue's own timestamp you cannot tell how old a number is, cannot filter on it, and cannot tell a real difference from a clock difference.
Store it on every row. Store your fetch time separately. Never conflate them. Detecting stale data.
Normalising across venues
Two venues express the same idea with different shapes: different outcome naming, different price scales, different threshold conventions, different structures for multi-outcome questions.
Write the normalisation as tested code with a documented intermediate representation, not as assumptions scattered through your analysis. Normalising across venues.
Being a good client
Identify yourself honestly in a user agent.
Respect documented rate limits and back off exponentially on errors rather than retrying immediately.
Cache what does not change. Resolution criteria and close times are nearly static.
Poll proportionally to interest. Frequent for a small watchlist, rare for the full market list. Rate limits and polling.
Prefer documented endpoints over scraping. And read the current terms of service, which change.
None of this is only politeness. A pipeline that behaves well keeps working.
Handling failure
Endpoints go down, rate-limit you, return partial data and occasionally return nonsense. A pipeline that assumes success produces a database with silent gaps, which is worse than one that stops.
Record failures as rows. Alert when a source has produced nothing for longer than expected, which is a different condition from producing values that are not changing. Alerting on price moves.
Next
Historical price series and their peculiarities.
Educational content only. Nothing here is financial advice, a solicitation, or an offer to trade. Prediction markets can move against you and a position can lose its entire value. Where this guide describes a difference between venues it describes a theoretical cross-market edge in observed prices, which is not a profit: fees, spreads, capital lockup, transfer time and differing settlement rules all come out of it first.
Keep reading
Data and Automation · 3 min readBacktesting a Market Signal HonestlyAlmost every impressive prediction market backtest is built on prices nobody could have traded. The specific errors, and how to build one that tells the truth.
Data and Automation · 3 min readHistorical Price Series and Their PeculiaritiesEvent contract history is not like equity history. Fixed bounds, a settlement point, irregular trading and a survivorship problem built into the data.
Data and Automation · 3 min readStoring Market Time Series Without Drowning in DataMost of what a naive collector records says nothing happened. Sampling, change-only writes and sensible retention keep a useful history small.
Polymarket · 3 min readThings That Surprise People on PolymarketWording that decides outcomes, capital stuck after the event is over, and the security rules that are absolute. The catch-out list in one place.