mlsanityData to 7 Oct 2026

Domyn-Small: A European 10B Reasoning Language Model

arXiv 2607.20448 ↗First posted 13 May 2026cs.CL cs.LGcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

We introduce Domyn-Small, a 10-billion-parameter open-weight reasoning language model released under the MIT license. Domyn-Small is the product of an initial pre-training phase on 9 trillion tokens multilingual data, followed by a post-training pipeline for reasoning, instruction following, and context extension. For the latter, we performed a Continued Pre-Training (CPT) phase that doubles the native context window to 32K tokens, followed by SFT with a math-focused annealing run. Finally, the RL phase includes GRPO with verifiable rewards, DPO, and a multi-environment GRPO stage spanning five task domains: mathematics, code, multiple-choice QA, instruction-following, and tool calling. The 32K-token native context extends to 128K at inference via YaRN, and a chat-template toggle enables dual-mode reasoning. Against peer models in the 7--10B class (Qwen3.5-9B, OLMo-3-7B-Think, Nemotron-Nano-8B, Ministral-3-8B), Domyn-Small achieves a strong accuracy-efficiency balance: it produces roughly one-third as many tokens as Qwen3.5-9B and approximately 35% of OLMo-3-7B-Think's token budget on core reasoning benchmarks, while delivering strong instruction-following (IFEval 79.9) and competitive science reasoning (GPQA-Diamond 50.0). We release the weights and the post-training recipe alongside Domyn Swarm (Apache~2.0), an open-source framework for scalable LLM inference on HPC clusters developed during this program and used throughout this work.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 22 of its 66 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

22 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Qwen3 Technical Report14 May

Other references · 21

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2016Layer Normalizationcited by 5,579
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  11. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  12. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
9 more
  1. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  2. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  3. 2024A StrongREJECT for Empty Jailbreakscited by 352
  4. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  5. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  6. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  7. 2025Olmo 3cited by 95
  8. 2024CyberMetric: A Benchmark Dataset based on Retrieval-Augmented Generation for Evaluating LLMs in Cybersecurity Knowledgecited by 55
  9. 2023SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Securitycited by 37

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Effective Learning for Small Reasoning Models: An Empirical Study on 0.5B Reasoning LLMs
  2. 2026Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning
  3. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning
  4. 2025MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes
  5. 2025Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math
  6. 2024LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning
  7. 2025Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
  8. 2024TinyThinker: Distilling Reasoning through Coarse-to-Fine Knowledge Internalization with Self-Reflection