mlsanityData to 7 Oct 2026

Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

arXiv 2512.20848 ↗First posted 23 Dec 2025cs.CL cs.AI cs.LGcited by 85

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 3 more
Abstract (arXiv)

We present Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer language model. Nemotron 3 Nano was pretrained on 25 trillion text tokens, including more than 3 trillion new unique tokens over Nemotron 2, followed by supervised fine tuning and large-scale RL on diverse environments. Nemotron 3 Nano achieves better accuracy than our previous generation Nemotron 2 Nano while activating less than half of the parameters per forward pass. It achieves up to 3.3x higher inference throughput than similarly-sized open models like GPT-OSS-20B and Qwen3-30B-A3B-Thinking-2507, while also being more accurate on popular benchmarks. Nemotron 3 Nano demonstrates enhanced agentic, reasoning, and chat abilities and supports context lengths up to 1M tokens. We release both our pretrained Nemotron 3 Nano 30B-A3B Base and post-trained Nemotron 3 Nano 30B-A3B checkpoints on Hugging Face.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 59 of its 74 references that are in the map. First posted on arXiv 23 Dec 2025.

Builds on

59 of 74 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model20 Aug
  2. 2025GenSelect: A Generative Approach to Best-of-N23 Jul
  3. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models4 Apr

Other references · 56

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  9. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  10. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  12. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
38 more
  1. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  2. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  3. 2021Measuring Coding Challenge Competence With APPScited by 879
  4. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  5. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  6. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  7. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  8. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  9. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  10. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  11. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  12. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  13. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  14. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  15. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
  16. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  17. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 191
  18. 2025Generalizing Verifiable Instruction Followingcited by 178
  19. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 171
  20. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
  21. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
  22. 2025Skywork Open Reasoner 1 Technical Reportcited by 161
  23. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 153
  24. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  25. 2023TACO: Topics in Algorithmic COde generation datasetcited by 150
  26. 2025RM-R1: Reward Modeling as Reasoningcited by 149
  27. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
  28. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  29. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 122
  30. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  31. 2025WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialectscited by 79
  32. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 75
  33. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 73
  34. 2025HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languagescited by 71
  35. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 57
  36. 2025Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Modelcited by 50
  37. 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Settingcited by 47
  38. 2025Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Datasetcited by 34

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 85.

85 papers.

Sort:

By area:

  1. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 33
  2. 2026OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesiscited by 32
  3. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 32
  4. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 20
  5. 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningcited by 13
  6. 2026Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacentercited by 11
  7. 2026SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoningcited by 10
  8. 2026Where does output diversity collapse in post-training?cited by 10
  9. 2026To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Modelscited by 9
  10. 2026PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decodingcited by 7
  11. 2026LOGIGEN: Logic-Driven Generation of Verifiable Agentic Taskscited by 5
  12. 2026Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Modelscited by 5
38 more
  1. 2026Aletheia: What Makes RLVR For Code Verifiers Tick?cited by 4
  2. 2026MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modelingcited by 4
  3. 2026ZAYA1-8B Technical Reportcited by 4
  4. 2026Masking Stale Observations Helps Search Agents -- Until It Doesn't: A Regime Map and Its Mechanismcited by 4
  5. 2026End-to-End Context Compression at Scalecited by 4
  6. 2026DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Modelscited by 4
  7. 2026Loop the Loopies!cited by 4
  8. 2026Raven: High-Recall Sequence Modeling with Sparse Memory Routingcited by 4
  9. 2026Effective Distillation to Hybrid xLSTM Architecturescited by 3
  10. 2026Reliable Chain-of-Thought via Prefix Consistencycited by 3
  11. 2026Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-trainingcited by 2
  12. 2026ABCD: All Biases Come Disguisedcited by 2
  13. 2026TriageSim: A Conversational Emergency Triage Simulation Framework from Structured Electronic Health Recordscited by 2
  14. 2026P^2O: Joint Policy and Prompt Optimizationcited by 2
  15. 2026Evaluating LLM-Based Test Generation Under Software Evolutioncited by 2
  16. 2026The ATOM Report: Measuring the Open Language Model Ecosystemcited by 2
  17. 2026PersonalHomeBench: Evaluating Agents in Personalized Smart Homescited by 2
  18. 2026SciImpact: A Multi-Dimensional, Multi-Field Benchmark for Scientific Impact Predictioncited by 2
  19. 2026SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inferencecited by 2
  20. 2026Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Controlcited by 2
  21. 2026A-Evolve-Training: Autonomous Post-Training of a 30B Modelcited by 2
  22. 2026Neglected Free Lunch from Post-training: Progress Advantage for LLM Agentscited by 2
  23. 2026Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigmscited by 2
  24. 2026CLIMP: Contrastive Language-Image Mamba Pretrainingcited by 1
  25. 2026LLM Compression by Block Removal with Constrained Binary Optimizationcited by 1
  26. 2026Kelix Technical Reportcited by 1
  27. 2026Evaluating LLM-Based Grant Proposal Review via Structured Perturbationscited by 1
  28. 2026Beyond Social Pressure: Benchmarking Epistemic Attack in Large Language Modelscited by 1
  29. 2026Normalized Architectures are Natively 4-Bitcited by 1
  30. 2026Laguna M.1/XS.2 Technical Reportcited by 1
  31. 2026On Subquadratic Architectures: From Applications to Principlescited by 1
  32. 2026MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysiscited by 1
  33. 2026VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Modelscited by 1
  34. 2026TW-LegalBench: Measuring Taiwanese Legal Understandingcited by 1
  35. 2026A Verifiable Search Is Not a Learnable Chain-of-Thoughtcited by 1
  36. 2026Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMscited by 1
  37. 2026TACTIC-KG: Toward Small Agent Teams for Cyber Threat Intelligence Knowledge Graph Constructioncited by 1
  38. 2026Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RLcited by 1

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025NVIDIA Nemotron 3: Efficient and Open Intelligence
  2. 2025Llama-Nemotron: Efficient Reasoning Models
  3. 2025Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
  4. 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence
  5. 2026HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model
  6. 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
  7. 2026Jupiter-N Technical Report
  8. 2025Apriel-Nemotron-15B-Thinker