# Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search

arXiv: https://arxiv.org/abs/2508.15884 · HTML: https://mlsanity.com/paper/2508.15884/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 91 of its 106 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 30 of 91)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)

## Built on by (most cited first, 30 of 41)

- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Kimi Linear: An Expressive, Efficient Attention Architecture](https://mlsanity.com/paper/2510.26692/) (2025)
- [Distilling to Hybrid Attention Models via KL-Guided Layer Selection](https://mlsanity.com/paper/2512.20569/) (2025)
- [Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts](https://mlsanity.com/paper/2601.22156/) (2026)
- [DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder](https://mlsanity.com/paper/2509.25182/) (2025)
- [Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models](https://mlsanity.com/paper/2511.18890/) (2025)
- [Hybrid Architectures for Language Models: Systematic Analysis and Design Insights](https://mlsanity.com/paper/2510.04800/) (2025)
- [Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow](https://mlsanity.com/paper/2601.14243/) (2026)
- [DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space](https://mlsanity.com/paper/2509.25180/) (2025)
- [Composer: A Search Framework for Hybrid Neural Architecture Design](https://mlsanity.com/paper/2510.00379/) (2025)
- [UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs](https://mlsanity.com/paper/2512.03383/) (2025)
- [Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design](https://mlsanity.com/paper/2511.17127/) (2025)
- [MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling](https://mlsanity.com/paper/2602.11761/) (2026)
- [Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design](https://mlsanity.com/paper/2605.15871/) (2026)
- [Erase-then-Delta Attention: Decoupling Erase and Write Addresses in Delta-Rule Linear Attention](https://mlsanity.com/paper/2606.26560/) (2026)
- [MDN: Parallelizing Stepwise Momentum for Delta Linear Attention](https://mlsanity.com/paper/2605.05838/) (2026)
- [What Limits Agentic Systems Efficiency?](https://mlsanity.com/paper/2510.16276/) (2025)
- [Distill-then-Replace: Efficient Task-Specific Hybrid Attention Model Construction](https://mlsanity.com/paper/2601.11667/) (2026)
- [When to Think Fast and Slow? AMOR: Adaptive Entropy Gate for Hybrid Models](https://mlsanity.com/paper/2602.13215/) (2026)
- [Apriel-H1: Towards Efficient Enterprise Reasoning Models](https://mlsanity.com/paper/2511.02651/) (2025)
- [AllMem: A Memory-centric Recipe for Efficient Long-context Modeling](https://mlsanity.com/paper/2602.13680/) (2026)
- [SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference](https://mlsanity.com/paper/2604.22575/) (2026)
- [Morphing into Hybrid Attention Models](https://mlsanity.com/paper/2606.30562/) (2026)
- [MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment](https://mlsanity.com/paper/2603.15954/) (2026)
- [When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models](https://mlsanity.com/paper/2603.26556/) (2026)
- [E3-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models](https://mlsanity.com/paper/2511.17205/) (2025)
- [LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs](https://mlsanity.com/paper/2604.22050/) (2026)
- [Long-Context Aware Upcycling: A New Frontier for Hybrid LLM Scaling](https://mlsanity.com/paper/2604.24715/) (2026)
- [HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation](https://mlsanity.com/paper/2605.17093/) (2026)
- [LLMForge: Multi-Backend Hardware-Aware Neural Architecture Search with Infinite-Head Attention for Edge Language Models](https://mlsanity.com/paper/2605.17653/) (2026)

## Abstract (arXiv)

We present Jet-Nemotron, a new family of hybrid-architecture language models, which matches or exceeds the accuracy of leading full-attention models while significantly improving generation throughput. Jet-Nemotron is developed using Post Neural Architecture Search (PostNAS), a novel neural architecture exploration pipeline that enables efficient model design. Unlike prior approaches, PostNAS begins with a pre-trained full-attention model and freezes its MLP weights, allowing efficient exploration of attention block designs. The pipeline includes four key components: (1) learning optimal full-attention layer placement and elimination, (2) linear attention block selection, (3) designing new attention blocks, and (4) performing hardware-aware hyperparameter search. Our Jet-Nemotron-2B model achieves comparable or superior accuracy to Qwen3, Qwen2.5, Gemma3, and Llama3.2 across a comprehensive suite of benchmarks while delivering up to 53.6x generation throughput speedup and 6.1x prefilling speedup. It also achieves higher accuracy on MMLU and MMLU-Pro than recent advanced MoE full-attention models, such as DeepSeek-V3-Small and Moonlight, despite their larger scale with 15B total and 2.2B activated parameters.