The Price of Thinking: Reasoning Effort as a Model-Specific API Contract

Details

Developers using reasoning models face an underdocumented API contract: the same model name under different reasoning-effort settings may behave differently in cost and performance. This paper measures that gap with a preregistered design — 30 AIME 2026 items, five calls each, comparing explicit "high effort" against the default omitted setting for Sonnet 5. The cost result is clear: the explicit-high contract adds a mean of $0.01031 per call over the omitted contract (95% CI: +$0.00204 to +$0.01974). The accuracy result is less clear: the observed difference was +0.0133, with a 95% CI of [-0.0267, +0.0467]. No statistically significant accuracy improvement was detected, though the interval cannot rule out a gain of up to 4.67 percentage points. Cost per correct answer was $0.08665 under the high-effort contract versus $0.07662 under the omitted contract — meaning omitted effort was slightly more cost-efficient as a point estimate. The paper also documents that omission semantics vary across providers and sometimes within the same provider, making "reasoning effort omitted" a model-version-specific behavior, not a universal default. For API buyers, the practical takeaway is that the cost of explicit high effort is real and consistent, while the accuracy benefit is uncertain and task-dependent. Teams optimizing for cost-per-correct-answer should validate reasoning-effort contracts on their specific task distribution before committing.