mlsanityData to 14 Oct 2026

Mercury: Ultra-Fast Language Models Based on Diffusion

arXiv 2506.17298 ↗First posted 17 Jun 2025cs.CL cs.AI cs.LGcited by 235

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

Abstract (arXiv)

We present Mercury, a new generation of commercial-scale large language models (LLMs) based on diffusion. These models are parameterized via the Transformer architecture and trained to predict multiple tokens in parallel. In this report, we detail Mercury Coder, our first set of diffusion LLMs designed for coding applications. Currently, Mercury Coder comes in two sizes: Mini and Small. These models set a new state-of-the-art on the speed-quality frontier. Based on independent evaluations conducted by Artificial Analysis, Mercury Coder Mini and Mercury Coder Small achieve state-of-the-art throughputs of 1109 tokens/sec and 737 tokens/sec, respectively, on NVIDIA H100 GPUs and outperform speed-optimized frontier models by up to 10x on average while maintaining comparable quality. We discuss additional results on a variety of code benchmarks spanning multiple languages and use-cases as well as real-world validation by developers on Copilot Arena, where the model currently ranks second on quality and is the fastest model overall. We also release a public API at https://platform.inceptionlabs.ai/ and free playground at https://chat.inceptionlabs.ai

Key facts. Cited by 235 papers in the map (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 31 of its 36 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

31 of 36 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2024The Llama 3 Herd of Modelscited by 14,715
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  10. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  12. 2022Scalable Diffusion Models with Transformerscited by 6,054
19 more
  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  8. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  10. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  11. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  12. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  13. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
  14. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  15. 2023Likelihood-Based Diffusion Language Modelscited by 151
  16. 2022MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generationcited by 136
  17. 2025The Amazon Nova Family of Models: Technical Report and Model Cardcited by 41
  18. 2025Copilot Arena: A Platform for Code LLM Evaluation in the Wildcited by 26
  19. 2025Enabling Autoregressive Models to Fill In Masked Tokenscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 81; 2026: 154.

235 papers.

Sort:

By area:

  1. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 178
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 116
  6. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  7. 2025A Survey on Diffusion Language Modelscited by 78
  8. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Modelscited by 71
  9. 2025Diffusion Language Models Know the Answer Before Decodingcited by 70
  10. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  11. 2025Any-Order Flexible Length Masked Diffusioncited by 51
  12. 2025Diffusion Language Models are Super Data Learnerscited by 48
38 more
  1. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 43
  2. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 41
  3. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  4. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 37
  5. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 34
  6. 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 34
  7. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 33
  8. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 33
  9. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  10. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  11. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimationcited by 27
  12. 2026Scaling Beyond Masked Diffusion Language Modelscited by 27
  13. 2025Training Optimal Large Diffusion Language Modelscited by 25
  14. 2025Optimal Inference Schedules for Masked Diffusion Modelscited by 25
  15. 2026DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inferencecited by 24
  16. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 24
  17. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 23
  18. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Modelscited by 23
  19. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagationcited by 21
  20. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 21
  21. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 20
  22. 2025Attention Sinks in Diffusion Language Modelscited by 20
  23. 2025Empirical Analysis of Decoding Biases in Masked Diffusion Modelscited by 19
  24. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  25. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 19
  26. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 19
  27. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Languagecited by 19
  28. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  29. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Modelscited by 18
  30. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 18
  31. 2025FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Modelscited by 17
  32. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Stepcited by 17
  33. 2026D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Draftingcited by 17
  34. 2025Inpainting-Guided Policy Optimization for Diffusion Large Language Modelscited by 16
  35. 2025Exploring the Potential of Diffusion Large Language Models in Code Generationcited by 16
  36. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 16
  37. 2025Soft-Masked Diffusion Language Modelscited by 16
  38. 2025Unveiling the Potential of Diffusion Large Language Model in Controllable Generationcited by 15

50 most cited of 235 All 235 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Mercury: A Code Efficiency Benchmark for Code Large Language Models
  2. 2026Alpha Diffusion Language Models: Factorization Alone Is Not the Problem
  3. 2025Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
  4. 2026Diffusion Language Models: An Experimental Analysis
  5. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusion
  6. 2025Large Language Models to Diffusion Finetuning
  7. 2025Constrained Decoding of Diffusion LLMs with Context-Free Grammars
  8. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding