mlsanityData to 7 Oct 2026

Xmodel-2 Technical Report

arXiv 2412.19638 ↗First posted 27 Dec 2024cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Xmodel-2 is a 1.2-billion-parameter large language model designed specifically for reasoning tasks. Its architecture enables different model scales to share a unified set of hyperparameters, allowing for extensive experimentation on smaller models and seamless transfer of optimal configurations to larger models. To maximize training efficiency and stability, Xmodel-2 employs the WSD learning rate scheduler from MiniCPM. Pretrained on 1.5 trillion tokens from diverse sources, Xmodel-2 achieves state-of-the-art performance in complex reasoning and agent-based tasks, while maintaining low training costs. These results highlight the potential of efficient model design and training strategies in advancing reasoning capabilities. Model checkpoints and code are publicly available on GitHub at https://github.com/XiaoduoAILab/Xmodel-2

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 27 of its 49 references that are in the map. First posted on arXiv 27 Dec 2024.

Builds on

27 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct

Other references · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2024Qwen2 Technical Reportcited by 2,076
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  9. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  10. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  11. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  12. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
14 more
  1. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  2. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  3. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  4. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  5. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  6. 2022The Stack: 3 TB of permissively licensed source codecited by 354
  7. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 313
  8. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 272
  9. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  10. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 216
  11. 2024Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performancecited by 179
  12. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  13. 2024Small Language Models: Survey, Measurements, and Insightscited by 109
  14. 2023Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networkscited by 94

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Xmodel-2.5: 1.3B Data-Efficient Reasoning SLM
  2. 2025Text2World: Benchmarking Large Language Models for Symbolic World Model Generation
  3. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models
  4. 2024LLM2: Let Large Language Models Harness System 2 Reasoning
  5. 2025A Survey of Scaling in Large Language Model Reasoning
  6. 2026X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes
  7. 2025Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router
  8. 2025Computational Reasoning of Large Language Models