mlsanityData to 7 Oct 2026

ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding

arXiv 2512.13586 ↗First posted 15 Dec 2025cs.CL cs.AI cs.LGcited by 16

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Autoregressive models (ARMs) are hindered by slow sequential inference. While masked diffusion models (MDMs) offer a parallel alternative, they suffer from critical drawbacks: high computational overhead from precluding Key-Value (KV) caching, and incoherent generation arising from learning dependencies over an intractable space of token combinations. To address these limitations, we introduce textscReFusion, a novel masked diffusion model that integrates sequence reorganization into the causal attention framework. By elevating parallel decoding from the token level to a higher slot level, textscReFusion interleaves inter-slot diffusion-based selection with intra-slot autoregressive infilling, while reordering newly generated slots ahead of the remaining masks after each iteration. Consequently, this design simultaneously unlocks full KV cache reuse and reduces learning complexity from an intractable token combination space to a manageable slot-level permutation space. Extensive experiments on seven diverse benchmarks show that textscReFusion not only overwhelmingly surpasses prior MDMs with a 34% performance gain and an over 18× speedup on average, but also bridges the performance gap to strong ARMs while maintaining a 2.33× average speedup.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 42 of its 46 references that are in the map. First posted on arXiv 15 Dec 2025.

Builds on

42 of 46 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs31 Aug
  2. 2025Dream 7B: Diffusion Large Language Models21 Aug
  3. 2025Empirical Analysis of Decoding Biases in Masked Diffusion Models18 Aug
  4. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models12 Aug
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  6. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models23 Jun
  7. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  8. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  9. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  10. 2025Large Language Diffusion Models14 Feb
  11. 2025Theoretical Benefit and Limitation of Diffusion Language Model13 Feb
  12. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
2 more
  1. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  2. 2023Diffusion Models for Non-autoregressive Text Generation: A Survey12 Mar

Other references · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2023Let's Verify Step by Stepcited by 3,598
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
16 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  2. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  3. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  4. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  5. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  6. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  7. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  8. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  9. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  10. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  11. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  12. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  13. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 64
  14. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 33
  15. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  16. 2024StreamingDialogue: Prolonged Dialogue Learning via Long Context Compression with Minimal Lossescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 16.

16 papers.

Sort:

By area:

  1. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  2. 2026CORE: Context-Robust Remasking for Diffusion Language Modelscited by 10
  3. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Accelerationcited by 8
  4. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculationcited by 5
  5. 2026Pythagoras-Prover: Advancing Efficient Formal Proving via Augmented Lean Formalisationcited by 3
  6. 2026Plan, Verify and Fill: A Structured Parallel Decoding Approach for Diffusion Language Modelscited by 2
  7. 2026DOS: Dependency-Oriented Sampler for Masked Diffusion Language Modelscited by 2
  8. 2026Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Modelscited by 1
  9. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  10. 2026LLaDA MoE v2: Scaling Mixture-of-Experts Diffusion Language Modelscited by 1
  11. 2026LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detectioncited by 0
  12. 2026Bifocal Diffusion Language Models: Asymmetric Bidirectional Context for Parallel Generationcited by 0
4 more
  1. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 0
  2. 2026Zarya: A Hybrid Autoregressive--Masked Diffusion Language Model with Flexible Training and Dual-Mode Inferencecited by 0
  3. 2026d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Modelscited by 0
  4. 2026Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Auto-Regressive Masked Diffusion Models
  2. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
  3. 2026Causal Autoregressive Diffusion Language Model
  4. 2026Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
  5. 2026Affix Cache for Diffusion Large Language Models
  6. 2026Edit-Based Refinement for Parallel Masked Diffusion Language Models
  7. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
  8. 2026Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale