mlsanityData to 14 Oct 2026

Mind the Cap: Output-Budget Regimes Change the Measured Multilingual Reasoning Gap

arXiv 2608.04160 ↗First posted 4 Aug 2026cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Multilingual evaluations report accuracy at a single output-token cap, but languages need different numbers of tokens to express the same content, so the cap is a hidden experimental variable. We test whether the native-vs-translate gap on MGSM (German, Thai, Swahili) is a token-budget artifact for Qwen3-8B and Llama-3.1-8B-Instruct under four prompting strategies. The measured gap swings by up to 57 points across budgets, length normalization moves it by up to 38.9 points where the cap binds, and at tight caps normalization can reverse which strategy scores higher. We prospectively froze the sweep's three Qwen peaks and its near-zero value at 1024 and evaluated them on 540,000 independently hard-capped decodes: a second frozen family of six Holm-corrected tests rejects every null. The frozen test at B^*=1024 still fails to reject because native accuracy has already saturated there; above saturation, the residual difference is a strategy-performance gap, not an identified reasoning deficit. The same truncation channel prices a cost-ordered adaptation ladder: a cross-fitted Thai vocabulary extension closes 0.0 points of the gap at the frozen budget and 4.9 points where 19% of traces still truncate. A third frozen family varies only the announced budget at a fixed enforced cap; announcing 128 rather than 2048 tokens moves Thai native accuracy by 5.1 points, so accuracy is not a function of the enforced cap alone. A correct-emission timing identity computed from one long-cap run matches the three pre-specified MGSM peaks to 0.65 points and, in an exploratory Qwen-only analysis of three further benchmarks, tracks held-out items to 0.92 points, locating the peak exactly in five of seven cells. Treat the output cap as an independent variable and report accuracy across the budget regime, not at a single budget.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 23 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

12 of 23 references.

References · 12

  1. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  2. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  3. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  4. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  5. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
  6. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  7. 2025Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gapcited by 43
  8. 2025MMATH: A Multilingual Benchmark for Mathematical Reasoningcited by 17
  9. 2026Crosslingual On-Policy Self-Distillation for Multilingual Reasoningcited by 10
  10. 2026Broken Chains: The Cost of Incomplete Reasoning in LLMscited by 4
  11. 2026Learning from Partial Chain-of-Thought via Truncated-Reasoning Self-Distillationcited by 3
  12. 2026Rethinking the Multilingual Reasoning Gap with Layer Swapcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.