AI & Development
Reasoning models like o3 and Claude's extended thinking generate internal reasoning before responding.
Reasoning models represent a distinct approach to improving LLM capability: instead of scaling model size or training data, they allocate more computation at inference time by generating extended chains of thought before producing a final response. OpenAI's o-series models and the extended thinking feature in Anthropic's Claude models are the leading implementations of this approach. The results on complex reasoning tasks are striking, and the trade-offs are significant enough that understanding them is necessary for deciding when to use these models.
A standard LLM generates a response token by token, moving from the prompt to the output in a single forward pass through the generation. A reasoning model inserts an additional step: before generating the visible response, it generates a hidden reasoning trace - an extended internal monologue where it works through the problem, considers alternatives, checks its reasoning, and arrives at a conclusion. The visible response is then generated based on both the original prompt and the reasoning trace.
The reasoning trace is typically hidden from the end user (shown in some interfaces as "thinking..." and collapsed by default) but is not hidden from the model - it is part of the context the model uses when generating the final response. The quality of the final response benefits from the reasoning process even if the reasoning itself is not shown.
The performance gains from reasoning models are concentrated in tasks that require multi-step reasoning, complex problem decomposition, or careful consideration of edge cases. Mathematical problem solving, advanced coding tasks, logical inference, and complex planning all show very large improvements when comparing reasoning models to comparable non-reasoning models. On tasks that do not require this depth of reasoning - simple question answering, text extraction, format conversion - reasoning models perform similarly to non-reasoning models but at significantly higher latency and cost.
Benchmarks consistently show reasoning models at the frontier of performance on mathematical and scientific reasoning tasks. For applications in these domains - tutoring systems, scientific analysis tools, complex code generation, research assistance - reasoning models are the appropriate choice.
Reasoning models are significantly more expensive than non-reasoning models due to the tokens consumed during the reasoning process. A reasoning trace might be 1,000-5,000 tokens long before the visible response is generated. This adds to the input token count (which is billed) and extends time-to-first-token (which affects latency). For tasks where the reasoning model's improvement over a standard model is substantial, this cost is justified. For tasks where the improvement is marginal, it is waste.
Thinking budget controls - available in some models - let you set a maximum number of thinking tokens, trading off reasoning depth against cost and latency. For interactive applications where latency matters, a smaller thinking budget produces faster responses at some quality cost. For batch processing or tasks where latency is acceptable, larger thinking budgets extract more performance.
Reasoning models are overkill for most conversational and instructional tasks. A customer support chatbot does not need extended reasoning to answer questions about return policies. A content summarization pipeline does not need 3,000 thinking tokens to summarize a 500-word document. Using reasoning models for simple tasks wastes money and adds latency without improving quality.
The right heuristic: use reasoning models when the task requires steps that are individually difficult to reason about, when the correct answer requires keeping track of multiple constraints simultaneously, or when previous attempts with standard models produced incorrect reasoning on similar tasks. For everything else, standard models provide better value.
One of the most effective uses of reasoning models is in planning steps of agentic workflows. An agent that needs to decompose a complex task into sub-tasks, reason about dependencies, and plan an execution order benefits substantially from extended thinking. The planning step may use a reasoning model, while the execution steps use faster, cheaper standard models for the actual tool calls and sub-task completion. This mixed-model architecture captures the reasoning benefits where they matter most without paying the reasoning premium on every inference call.