Reasoning Models: o1/o3/R1-Style Models and Test-Time Compute
~13 min read
A newer class of models is trained to generate long internal reasoning by default, rather than relying on a human's prompt to request it — trading more inference-time compute for better performance on hard reasoning tasks.
Reasoning Models: o1/o3/R1-Style Models and Test-Time Compute is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Reasoning models (o1/o3, R1-style) are trained to reason well by default, rather than relying on a human's prompt (like CoT) to request reasoning steps
- •They're trained with additional RL on verifiable rewards (math/code/logic correctness) — the same GRPO-style RFT approach from sft-vs-rft, applied to shape default behavior rather than being prompt-invoked
- •This connects to 'test-time compute': capability can be extended by spending more computation at INFERENCE time, not just more training-time compute — a reasoning model spends more 'thinking' on harder problems automatically
- •Many reasoning models don't show the full raw internal reasoning trace to the user (unlike CoT prompting, where the trace IS the visible output) — often summarized or hidden
- •The tradeoff: better performance on genuinely hard reasoning tasks, at higher latency/cost per query — a poor fit for simple queries, a strong fit for the hard/verifiable end of tasks