JS Denain responds to Toby Ord’s recent analysis of RL and inference scaling, which argued that RL scaling is dramatically inefficient (10,000x more compute than inference scaling) and that inference costs create a persistent economic burden. Denain pushes back on both claims, arguing that (1) the dollar cost to reach a fixed capability level falls rapidly — roughly 5-10x per year — through distillation, algorithmic improvements, and reduced token verbosity, and (2) the RL inefficiency estimate relies on very few data points and may already be outdated given algorithmic progress since o1/o3.

Key Insights

The most compelling evidence is the FrontierMath benchmark comparison: reaching ~27% accuracy required 43 million tokens with o4-mini but only 5 million tokens with GPT-5.2 eight months later — a ~3x cost reduction from fewer tokens alone, before accounting for per-token cost decreases. The broader point that multiple cost-reduction mechanisms stack (smaller models via distillation, cheaper per-token inference, fewer tokens needed) makes the 5-10x annual reduction estimate plausible, though Denain rightly flags this may decelerate. The critique of Ord’s 10,000x RL inefficiency figure as based on sparse data is fair — extrapolating scaling laws from “a small number of data points” across a rapidly moving field is inherently fragile.

Our Take

This piece is technically careful and makes a useful correction to what could become a misleading narrative about RL scaling being fundamentally uneconomical. The practical implication matters: if inference costs for a given capability level drop 5-10x annually, then today’s expensive reasoning models become tomorrow’s cheap commodity — which has direct implications for prediction markets on AI deployment timelines. The weakness is that Denain’s own estimates are also extrapolations from limited data, so this is really an argument about whose extrapolation is less wrong. Still, the direction of the argument (costs fall fast, RL may improve faster than current data suggests) aligns with historical patterns in ML efficiency gains.