mlsanityData to 14 Oct 2026

Energy Consumption Analysis of Discrete Diffusion and Autoregressive Language Models

arXiv 2610.19855 ↗First posted 14 Oct 2026cs.LGcited by 0

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) models are the industry standard, Diffusion Language Models (DLLMs) have emerged as a promising alternative capable of parallel token generation. In this paper, we perform a comparative energy benchmark of diffusion models (LLaDA-8B, Dream-7B) against AR counterparts (Llama-3.1-8B, Qwen-2.5-7B). We analyze the trade-offs between model architecture, KV-cache optimization, and Joules-per-token efficiency. Our results show that at longer sequences (200--600 tokens), diffusion models achieve 2.5--2.7 J/token, approximately 2× more efficient than AR models with KV-cache (5.3--5.7 J/token). KV-cache provides substantial benefits for AR models at longer sequences (50--70% energy reduction).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 6 of its 9 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

6 of 9 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  2. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct

Other references · 4

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  3. 2019Green AIcited by 627
  4. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  2. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
  3. 2025Dream 7B: Diffusion Large Language Models
  4. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
  5. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  6. 2025dKV-Cache: The Cache for Diffusion Language Models
  7. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  8. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models