mlsanityData to 7 Oct 2026

Capacity, Not Format: Rethinking Structured Reasoning Failures

arXiv 2606.09410 ↗First posted 8 Jun 2026cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Prior work treats structured output as a reasoning tax, but this framing is incomplete: the cost of formatting depends strongly on a model's spare capacity. Using information-matched prose controls and a four-level schema complexity gradient, we separate format-specific effects from prompt-length confounds across 4 models and 5 benchmarks with 0% parse failures on successfully generated responses. We find that structured formats are capacity-dependent. Models with sufficient headroom absorb JSON constraints without degradation (Sonnet: 88.7±4.0% JSON vs. 89.3±1.7% CoT on MATH-Hard). In contrast, formats severely degrade models operating near their limits through two distinct mechanisms. First, under standard token budgets, Haiku drops 36.2pp (p < 0.0001) largely due to truncation. Second, even with extended budgets eliminating truncation, GPT-4o-mini drops 28.0pp (p < 0.001), revealing pure capacity competition independent of token exhaustion. This format penalty scales with schema complexity (McNemar p < 0.0001) and cannot be explained by prompt length alone. Furthermore, these results qualify claims of frontier model immunity: on AIME competition math, Opus 4.7 drops from 96.2% to 91.0% under JSON (-5.3pp; the displayed percentages are independently rounded, exact difference is 7/133 = 5.26pp ≈ 5.3pp). A delayed-structure ablation -- reasoning freely before formatting -- recovers most of the lost accuracy (3-run mean: 80--87%), supporting the capacity competition mechanism. The practical implication is not to avoid structured output, but to match it to capacity: when a model is near its limits, think first, format later.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 6 of its 17 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

6 of 17 references.

References · 6

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  6. 2025Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inferencecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models
  2. 2026Where vs What: Decomposing Structural and Content Failures in LLM-Generated Structured Outputs
  3. 2026Breaking the Space Barrier and its Application to Language Model Inference
  4. 2025Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models
  5. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
  6. 2026Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap
  7. 2025Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression
  8. 2026The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits