mlsanityData to 14 Oct 2026

InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning

arXiv 2502.11573 ↗First posted 17 Feb 2025cs.CL cs.AIcited by 2

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have made significant advancements in reasoning capabilities. However, they still face challenges such as high computational demands and privacy concerns. This paper focuses on developing efficient Small Language Models (SLMs) and Multimodal Small Language Models (MSLMs) that retain competitive reasoning abilities. We introduce a novel training pipeline that enhances reasoning capabilities and facilitates deployment on edge devices, achieving state-of-the-art performance while minimizing development costs. InfR~ aims to advance AI systems by improving reasoning, reducing adoption barriers, and addressing privacy concerns through smaller model sizes. Resources are available at https://github. com/Reallm-Labs/InfiR.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 36 of its 44 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

36 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024MAmmoTH2: Scaling Instructions from the Web6 May
  2. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermism5 Jan
  3. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb

Other references · 33

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
21 more
  1. 2025s1: Simple test-time scalingcited by 1,341
  2. 2024OpenAI o1 System Cardcited by 1,244
  3. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  4. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  5. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  6. 2023StarCoder: may the source be with you!cited by 799
  7. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  8. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  9. 20242 OLMo 2 Furiouscited by 340
  10. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  11. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 155
  12. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  13. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
  14. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 95
  15. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  16. 2024HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalizationcited by 86
  17. 2024VISTA: Visualized Text Embedding For Universal Multi-Modal Retrievalcited by 67
  18. 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflectioncited by 67
  19. 2024InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoningcited by 36
  20. 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classificationcited by 35
  21. 2024Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratchcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Promptscited by 4
  2. 2026TRIAGE: Direction-Aware Mismatch Stabilization of Native NVFP4 Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models
  2. 2025Towards Reasoning Ability of Small Language Models
  3. 2023InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models
  4. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
  5. 2024Small Language Models: Survey, Measurements, and Insights
  6. 2025Effective Learning for Small Reasoning Models: An Empirical Study on 0.5B Reasoning LLMs
  7. 2025Guiding Reasoning in Small Language Models with LLM Assistance
  8. 2024Improving Small-Scale Large Language Models Function Calling for Reasoning Tasks