Running an Autonomous Trading Bot on a Prediction Market for 60 Days: What We Learned

We built and ran an autonomous forecasting agent on a real prediction market for 60 days. It started on paper trading, graduated to live capital, and eventually we shut it down. Here’s the honest postmortem.

The Hypothesis

Prediction markets are, in theory, efficient—prices should reflect the true probability of outcomes. In practice, they’re imperfect. Retail participants bet on intuition; news cycles create temporary mispricing; illiquid markets have wide spreads. An AI agent with access to current information and a calibrated sense of probability might find systematic edges.

The Architecture

The agent ran on a daily cycle. Each run it would: survey open markets in its coverage area (geopolitical events, economic indicators, weather-related outcomes), assess current prices against its own probability estimates, and place limit orders where the price-probability gap exceeded a threshold. It used a dedicated Python client for the market’s API and tracked all positions in a local ledger database.

Paper trading ran for several weeks with a simulated bankroll. When the win rate and P&L looked reasonable, we moved to live capital with real money at stake.

What Worked

The agent’s best performance was on geopolitical markets where it had strong priors and the market had thin liquidity. An early correct call on a diplomatic outcome—buying at a significant discount to what turned out to be the actual probability—generated a solid return. The agent was good at identifying markets where the crowd was clearly overconfident in a particular direction.

What Didn’t

Three consistent failure modes emerged:

• Stale information on fast-moving markets: When news breaks, prediction markets reprice within minutes. An agent running on a daily cycle is always behind. We lost money on several markets where the agent’s position was correct at entry time and wrong by the time it could adjust.
• Liquidity risk in small markets: Thin order books meant the agent’s own orders moved the market price, making entries worse than the displayed price suggested.
• Weather markets specifically: The agent consistently underperformed on weather-outcome markets. Weather modeling requires real-time meteorological data integration that goes well beyond news aggregation. We eventually shut down the weather strategy entirely.

The Decision to Stop

We stopped the live trading component not because it was a catastrophic failure but because the edge wasn’t consistent enough to justify ongoing capital risk. Paper trading P&L looked better than live P&L—a common pattern where simulation doesn’t fully capture execution risk. The experiment generated useful data about where AI forecasting agents have genuine edges (calibrated probability estimation on well-defined events) versus where they don’t (real-time reaction to fast-moving information).

What This Means for AI in Financial Contexts

AI agents in any financial context need real-time data access to compete with market participants who have it. A daily-cycle agent is structurally disadvantaged against humans and algorithms monitoring the same markets continuously. If you’re building AI for financial use cases, the data infrastructure is at least as important as the model. Getting the intelligence right without getting the data infrastructure right just means being wrong faster.