mlsanityData to 14 Oct 2026

Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See

arXiv 2608.17744 ↗First posted 18 Aug 2026cs.CL cs.LG cs.ROcited by 0

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 37 of its 42 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

37 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer23 Jan

Other references · 36

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  8. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  9. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  10. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  11. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  12. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
24 more
  1. 2024LoRA Learns Less and Forgets Lesscited by 328
  2. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
  3. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  4. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  5. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
  6. 2021Accounting for Variance in Machine Learning Benchmarkscited by 201
  7. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 191
  8. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  9. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 169
  10. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 165
  11. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  12. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
  13. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 97
  14. 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 96
  15. 2023Do Multilingual Language Models Think Better in English?cited by 88
  16. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 59
  17. 2024Quantifying Variance in Evaluation Benchmarkscited by 54
  18. 2023All Languages Matter: On the Multilingual Safety of Large Language Modelscited by 53
  19. 2024Rethinking Data Selection at Scale: Random Selection is Almost All You Needcited by 22
  20. 2025Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learningcited by 18
  21. 2025Krikri: Advancing Open Large Language Models for Greekcited by 14
  22. 2024A Comprehensive Survey of Contamination Detection Methods in Large Language Modelscited by 12
  23. 2026MORFES: A Benchmark for Productive Inflectional Competence in Modern Greekcited by 1
  24. 2026Teaching Nemotron Greek: Mining a Corpus, Adapting Retrieval, and Grounding Generation for Modern Greek across Specialist Domainscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models
  2. 2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
  3. 2025Input-Time Scaling: Adding Noise and Irrelevance into Less-Is-More Drastically Improves Reasoning Performance and Efficiency
  4. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
  5. 2025Large Language Models Imitate Logical Reasoning, but at what Cost?
  6. 2025Thought calibration: Efficient and confident test-time scaling
  7. 2025The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation
  8. 2026Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge