# What does FFN compression change downstream? Same-state causal restoration in diffusion language models

arXiv: https://arxiv.org/abs/2609.31685 · HTML: https://mlsanity.com/paper/2609.31685/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching](https://mlsanity.com/paper/2509.23094/) (2025)
- [Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs](https://mlsanity.com/paper/2508.14896/) (2025)
- [When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models](https://mlsanity.com/paper/2504.02010/) (2025)
- [Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models](https://mlsanity.com/paper/2510.03274/) (2025)
- [Universal Properties of Activation Sparsity in Modern Large Language Models](https://mlsanity.com/paper/2509.00454/) (2025)
- [Stopping Computation for Converged Tokens in Masked Diffusion-LM Decoding](https://mlsanity.com/paper/2602.06412/) (2026)
- [Layer Collapse in Diffusion Language Models](https://mlsanity.com/paper/2605.06366/) (2026)
- [TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models](https://mlsanity.com/paper/2606.12841/) (2026)
- [Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Experts](https://mlsanity.com/paper/2608.11212/) (2026)
- [Knowledge Editing for Masked Diffusion Language Models](https://mlsanity.com/paper/2606.03924/) (2026)

## Abstract (arXiv)

Diffusion language models (DLMs) enable flexible, parallel generation, but their iterative denoising remains computationally expensive, motivating increasingly aggressive compression. Existing compression objectives largely measure how well compressed computation approximates the original locally, but local error does not reveal which removed computations actually matter to the downstream denoising trajectory. We introduce Same-State Causal Restoration (SSR), which restores the original FFN on the exact current input reached by the compressed model and measures how the resulting trajectory changes. To our knowledge, this is the first direct measurement of the same-current-input closed-loop effect of removed FFN computation in DLM compression. Across LLaDA-8B-Instruct and Dream-v0-Instruct-7B, compressed-side state ranks this downstream effect substantially better than local NMSE at fixed denoising phase, while controlled interventions show that correction structure matters beyond magnitude. Using task-label-free calibration, SSR freezes a single restoration window for held-out inference. Under aggressive LLaDA compression, restoring only four transitions recovers 89.9% of the lost accuracy while retaining an estimated 36.8% whole-model MAC saving and outperforming an equal-budget local-error baseline. Dream further shows that restoring dense behavior and repairing the final task are distinct outcomes.