Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference
On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference
Abstract (arXiv)
Diffusion Large Language Models (DLLMs) promise fast non-autoregressive inference but suffer a severe quality-speed trade-off in parallel decoding. This stems from the ''combinatorial contradiction'' phenomenon, where parallel tokens form semantically inconsistent combinations. We address this by integrating continuous representations into the discrete decoding process, as they preserve rich inter-position dependency. We propose ReMix (Rejection Mixing), a framework that introduces a novel Continuous Mixing State as an intermediate between the initial masked state and the final decoded token state. This intermediate state allows a token's representation to be iteratively refined in a continuous space, resolving mutual conflicts with other tokens before collapsing into a final discrete sample. Furthermore, a rejection rule reverts uncertain representations from the continuous state back to the masked state for reprocessing, ensuring stability and preventing error propagation. ReMix thus mitigates combinatorial contradictions by enabling continuous-space refinement during discrete diffusion decoding. Extensive experiments demonstrate that ReMix, as a training-free method, achieves a 2-8 × inference speedup without any quality degradation.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 50 of its 54 references that are in the map. First posted on arXiv 26 Feb 2026.
Builds on
50 of 54 references.
Closest ancestors · 6
Same direction, newest first
- 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models30 Sep
- 2025dParallel: Learnable Parallel Decoding for dLLMs30 Sep
- 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction4 Aug
- 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding31 May
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
- 2025dKV-Cache: The Cache for Diffusion Language Models21 May
Other references · 44
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Visual Instruction Tuningcited by 7,792
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2021Program Synthesis with Large Language Modelscited by 3,509
32 more
- 2023A Survey of Large Language Modelscited by 2,537
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2025Large Language Diffusion Modelscited by 872
- 2024Simple and Effective Masked Diffusion Language Modelscited by 800
- 2016A Diagram Is Worth A Dozen Imagescited by 793
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
- 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
- 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
- 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
- 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
- 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
- 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
- 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
- 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
- 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
- 2025A Survey on Diffusion Language Modelscited by 75
- 2024Discrete Copula Diffusioncited by 53
- 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 45
- 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 43
- 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
- 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 31
- 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
- 2025Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithmscited by 12
- 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decodingcited by 6
- 2025Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigationcited by 5
- 2025HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Modelscited by 4
- 2026Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformerscited by 4
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Modelscited by 1
- 2026Accelerating Masked Image Generation by Learning Controlled Latent Dynamicscited by 0
- 2026Diffusion Language Model for Recommendationcited by 0
- 2026Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inferencecited by 0
- 2026Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference27 Aug 2026
- 2026Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Models12 Aug 2026
- 2026Diffusion Language Model for Recommendation23 Jul 2026
- 2026Accelerating Masked Image Generation by Learning Controlled Latent Dynamics27 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
- 2026Fast-dLLM++: Fr'echet Profile Decoding for Faster Diffusion LLM Inference
- 2026Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
- 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
- 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
- 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMs
- 2026Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
- 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching