mlsanityData to 7 Oct 2026

Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts

arXiv 2602.13367 ↗First posted 13 Feb 2026cs.AI cs.CLcited by 5

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 2 more
Abstract (arXiv)

We present Nanbeige4.1-3B, a unified generalist language model that simultaneously achieves strong agentic behavior, code generation, and general reasoning with only 3B parameters. To the best of our knowledge, it is the first open-source small language model (SLM) to achieve such versatility in a single model. To improve reasoning and preference alignment, we combine point-wise and pair-wise reward modeling, ensuring high-quality, human-aligned responses. For code generation, we design complexity-aware rewards in Reinforcement Learning, optimizing both correctness and efficiency. In deep search, we perform complex data synthesis and incorporate turn-level supervision during training. This enables stable long-horizon tool interactions, allowing Nanbeige4.1-3B to reliably execute up to 600 tool-call turns for complex problem-solving. Extensive experimental results show that Nanbeige4.1-3B significantly outperforms prior models of similar scale, such as Nanbeige4-3B-2511 and Qwen3-4B, even achieving superior performance compared to much larger models, such as Qwen3-30B-A3B. Our results demonstrate that small models can achieve both broad competence and strong specialization simultaneously, redefining the potential of 3B parameter models.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 12 of its 24 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

12 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models15 Dec

Other references · 11

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  3. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
  4. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 139
  5. 2025LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?cited by 52
  6. 2025ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Datasetcited by 27
  7. 2025A Unified Pairwise Framework for RLHF: Bridging Generative Reward Modeling and Policy Optimizationcited by 21
  8. 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningcited by 13
  9. 2026RE-TRAC: REcursive TRAjectory Compression for Deep Search Agentscited by 9
  10. 2025Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5Bcited by 8
  11. 2025Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Tiny Reward Models
  2. 2024CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning
  3. 2026Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
  4. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  5. 2025URPO: A Unified Reward & Policy Optimization Framework for Large Language Models
  6. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
  7. 2025NP-Engine: Empowering Optimization Reasoning in Large Language Models with Verifiable Synthetic NP Problems
  8. 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning