mlsanityData to 14 Oct 2026

What does FFN compression change downstream? Same-state causal restoration in diffusion language models

arXiv 2609.31685 ↗First posted 16 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Diffusion language models (DLMs) enable flexible, parallel generation, but their iterative denoising remains computationally expensive, motivating increasingly aggressive compression. Existing compression objectives largely measure how well compressed computation approximates the original locally, but local error does not reveal which removed computations actually matter to the downstream denoising trajectory. We introduce Same-State Causal Restoration (SSR), which restores the original FFN on the exact current input reached by the compressed model and measures how the resulting trajectory changes. To our knowledge, this is the first direct measurement of the same-current-input closed-loop effect of removed FFN computation in DLM compression. Across LLaDA-8B-Instruct and Dream-v0-Instruct-7B, compressed-side state ranks this downstream effect substantially better than local NMSE at fixed denoising phase, while controlled interventions show that correction structure matters beyond magnitude. Using task-label-free calibration, SSR freezes a single restoration window for held-out inference. Under aggressive LLaDA compression, restoring only four transitions recovers 89.9% of the lost accuracy while retaining an estimated 36.8% whole-model MAC saving and outperforming an equal-budget local-error baseline. Dream further shows that restoring dense behavior and repairing the final task are distinct outcomes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 14 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

12 of 14 references.

References · 12

  1. 2025Large Language Diffusion Modelscited by 912
  2. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  3. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 25
  4. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 23
  5. 2025When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Modelscited by 13
  6. 2025Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Modelscited by 12
  7. 2025Universal Properties of Activation Sparsity in Modern Large Language Modelscited by 7
  8. 2026Stopping Computation for Converged Tokens in Masked Diffusion-LM Decodingcited by 3
  9. 2026Layer Collapse in Diffusion Language Modelscited by 2
  10. 2026TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Modelscited by 2
  11. 2026Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Expertscited by 2
  12. 2026Knowledge Editing for Masked Diffusion Language Modelscited by 1

Built on by

Not cited yet.