mlsanityData to 13 Oct 2026

From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons

arXiv 2605.27387 ↗First posted 11 Apr 2026cs.CL cs.AIcited by 2

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion models promise efficient parallel text generation but rely on bidirectional attention, creating a structural mismatch with pre-trained Autoregressive (AR) models. This incompatibility precludes reusing robust AR priors, necessitating prohibitive pre-training from scratch. To bridge this gap, we propose FLUID, a framework that efficiently adapts AR backbones to the diffusion paradigm. By enforcing Strictly Causal Alignment, FLUID enables seamless initialization from standard GPT-style checkpoints, circumventing the need for massive pre-training. Furthermore, we introduce Elastic Horizons, an entropy-driven mechanism that dynamically modulates denoising strides based on local information density rather than fixed schedules. Experiments demonstrate that FLUID achieves state-of-the-art performance while reducing training costs by orders of magnitude, effectively reconciling established AR foundations with efficient parallel generation. Our code is available at https://github.com/Oli-lab-nun/FLUID/tree/main.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 22 of its 33 references that are in the map. First posted on arXiv 11 Apr 2026.

Builds on

22 of 33 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Scaling up Masked Diffusion Models on Text24 Oct
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  3. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation6 May
  4. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,486
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  4. 2023Let's Verify Step by Stepcited by 3,672
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,685
  6. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,012
  7. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 833
  8. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 409
  9. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 228
  10. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 207
  11. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  12. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 132
6 more
  1. 2025Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Modelscited by 50
  2. 2025TreeDiff: AST-Guided Code Generation with Diffusion LLMscited by 11
  3. 2025SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layerscited by 9
  4. 2025IAM: Efficient Inference through Attention Mapping between Different-scale LLMscited by 8
  5. 2025KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embeddingcited by 5
  6. 2025XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compressioncited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Modelscited by 0
  2. 2026Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
  2. 2025Diffusion Language Models are Super Data Learners
  3. 2026Don't Retrain, Align: Adapting Autoregressive LMs to Diffusion LMs via Representation Alignment
  4. 2026Reducing Pretraining-Generation Mismatch in Diffusion Language Models
  5. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  6. 2026Enhancing Diffusion Language Models with Autoregressive Post-Training Weights
  7. 2025Dream 7B: Diffusion Large Language Models
  8. 2025Large Language Models to Diffusion Finetuning