mlsanityData to 14 Oct 2026

Xmodel-2.5: 1.3B Data-Efficient Reasoning SLM

arXiv 2511.19496 ↗First posted 23 Nov 2025cs.LG cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models deliver strong reasoning and tool-use skills, yet their computational demands make them impractical for edge or cost-sensitive deployments. We present Xmodel-2.5, a 1.3-billion-parameter small language model designed as a drop-in agent core. Training with maximal-update parameterization (μP) allows hyper-parameters tuned on a 20M-parameter proxy to transfer directly to the full model, even under the parameter-tied tie-word-embedding architecture. A 1.4T-token Warmup--Stable--Decay curriculum is used, and we further show that switching from AdamW to Muon during the decay phase improves the 13-task reasoning average by 4.58,% while keeping every other hyper-parameter fixed, verifying that early AdamW stability can be paired with late Muon sharpening for better downstream performance. FP8-mixed-precision training balances accuracy and throughput. All checkpoints, recipes, and evaluation code are released under the Apache-2.0 license.footnotehttps://huggingface.co/XiaoduoAILab/Xmodel-2.5 and https://huggingface.co/XiaoduoAILab/Xmodel-2.5-history (training checkpoints). Training code and evaluation harness: https://github.com/XiaoduoAILab/Xmodel-2.5.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 17 of its 19 references that are in the map. First posted on arXiv 23 Nov 2025.

Builds on

17 of 19 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  9. 2021Program Synthesis with Large Language Modelscited by 3,637
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
5 more
  1. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  2. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 320
  3. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
  4. 2023Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networkscited by 97
  5. 2025MiniCPM4: Ultra-Efficient LLMs on End Devicescited by 52

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  2. 2026Efficient Reasoning on the Edge
  3. 2025Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
  4. 2026Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
  5. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
  6. 2025A Survey of Scaling in Large Language Model Reasoning
  7. 2026CosmicFish-HRM: Adaptive Reasoning via Hierarchical Recurrent Mechanisms in Compact Language Models
  8. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs