# Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

arXiv: https://arxiv.org/abs/2512.20848 · HTML: https://mlsanity.com/paper/2512.20848/

## Key facts

- Cited by 104 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 59 of its 74 references that are in the map.
- First posted on arXiv 23 Dec 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- Mixture-of-experts / [MoE in reinforcement learning and robotics](https://mlsanity.com/moe/moe-in-reinforcement-learning-and-robotics/)
- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 30 of 59)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models](https://mlsanity.com/paper/2401.06066/) (2024)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [DataComp-LM: In search of the next generation of training sets for language models](https://mlsanity.com/paper/2406.11794/) (2024)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)
- [JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2404.03027/) (2024)
- [Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts](https://mlsanity.com/paper/2408.15664/) (2024)
- [OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data](https://mlsanity.com/paper/2410.01560/) (2024)
- [Generalizing Verifiable Instruction Following](https://mlsanity.com/paper/2507.02833/) (2025)

## Built on by (most cited first, 30 of 104)

- [Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://mlsanity.com/paper/2606.15007/) (2026)
- [Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://mlsanity.com/paper/2604.12374/) (2026)
- [OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis](https://mlsanity.com/paper/2603.20278/) (2026)
- [Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery](https://mlsanity.com/paper/2601.20088/) (2026)
- [Olmo Hybrid: From Theory to Practice and Back](https://mlsanity.com/paper/2604.03444/) (2026)
- [Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning](https://mlsanity.com/paper/2601.09088/) (2026)
- [To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models](https://mlsanity.com/paper/2602.12566/) (2026)
- [SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning](https://mlsanity.com/paper/2602.22603/) (2026)
- [Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter](https://mlsanity.com/paper/2604.15039/) (2026)
- [Where does output diversity collapse in post-training?](https://mlsanity.com/paper/2604.16027/) (2026)
- [PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding](https://mlsanity.com/paper/2605.08632/) (2026)
- [Loop the Loopies!](https://mlsanity.com/paper/2607.16051/) (2026)
- [LOGIGEN: Logic-Driven Generation of Verifiable Agentic Tasks](https://mlsanity.com/paper/2603.00540/) (2026)
- [FLARE: Diffusion for Hybrid Language Model](https://mlsanity.com/paper/2606.01774/) (2026)
- [Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models](https://mlsanity.com/paper/2606.05688/) (2026)
- [Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents](https://mlsanity.com/paper/2606.26080/) (2026)
- [Raven: High-Recall Sequence Modeling with Sparse Memory Routing](https://mlsanity.com/paper/2607.25357/) (2026)
- [Aletheia: What Makes RLVR For Code Verifiers Tick?](https://mlsanity.com/paper/2601.12186/) (2026)
- [MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling](https://mlsanity.com/paper/2602.11761/) (2026)
- [Priming: Hybrid State Space Models From Pre-trained Transformers](https://mlsanity.com/paper/2605.08301/) (2026)
- [Masking Stale Observations Helps Search Agents -- Until It Doesn't: A Regime Map and Its Mechanism](https://mlsanity.com/paper/2606.00408/) (2026)
- [DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models](https://mlsanity.com/paper/2606.19257/) (2026)
- [ZAYA1-8B Technical Report](https://mlsanity.com/paper/2605.05365/) (2026)
- [End-to-End Context Compression at Scale](https://mlsanity.com/paper/2606.09659/) (2026)
- [Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms](https://mlsanity.com/paper/2608.27409/) (2026)
- [ABCD: All Biases Come Disguised](https://mlsanity.com/paper/2602.17445/) (2026)
- [Effective Distillation to Hybrid xLSTM Architectures](https://mlsanity.com/paper/2603.15590/) (2026)
- [P^2O: Joint Policy and Prompt Optimization](https://mlsanity.com/paper/2603.21877/) (2026)
- [Star Elastic: Many-in-One Reasoning LLMs with Efficient Budget Control](https://mlsanity.com/paper/2605.07182/) (2026)
- [Reliable Chain-of-Thought via Prefix Consistency](https://mlsanity.com/paper/2605.07654/) (2026)

## Abstract (arXiv)

We present Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer language model. Nemotron 3 Nano was pretrained on 25 trillion text tokens, including more than 3 trillion new unique tokens over Nemotron 2, followed by supervised fine tuning and large-scale RL on diverse environments. Nemotron 3 Nano achieves better accuracy than our previous generation Nemotron 2 Nano while activating less than half of the parameters per forward pass. It achieves up to 3.3x higher inference throughput than similarly-sized open models like GPT-OSS-20B and Qwen3-30B-A3B-Thinking-2507, while also being more accurate on popular benchmarks. Nemotron 3 Nano demonstrates enhanced agentic, reasoning, and chat abilities and supports context lengths up to 1M tokens. We release both our pretrained Nemotron 3 Nano 30B-A3B Base and post-trained Nemotron 3 Nano 30B-A3B checkpoints on Hugging Face.