# Preserving Mathematical Reasoning in Compressed Diffusion Language Models via Trajectory-Aware Low-Rank Approximation

arXiv: https://arxiv.org/abs/2610.03326 · HTML: https://mlsanity.com/paper/2610.03326/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning.
- Builds on 15 of its 15 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Placed in

- LLM reasoning / [Diffusion reasoning and test-time scaling](https://mlsanity.com/lr/diffusion-reasoning-and-test-time-scaling/)
- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 15 of 15)

- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Q-Diffusion: Quantizing Diffusion Models](https://mlsanity.com/paper/2302.04304/) (2023)
- [SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression](https://mlsanity.com/paper/2403.07378/) (2024)
- [ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models](https://mlsanity.com/paper/2312.05821/) (2023)
- [Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models](https://mlsanity.com/paper/2509.06949/) (2025)
- [MoDeGPT: Modular Decomposition for Large Language Model Compression](https://mlsanity.com/paper/2408.09632/) (2024)
- [Towards Accurate Post-training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.18723/) (2023)
- [Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models](https://mlsanity.com/paper/2510.03274/) (2025)
- [SADA: Stability-guided Adaptive Diffusion Acceleration](https://mlsanity.com/paper/2507.17135/) (2025)
- [DLLMQuant: Quantizing Diffusion-based Large Language Models](https://mlsanity.com/paper/2508.14090/) (2025)
- [FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models](https://mlsanity.com/paper/2508.01506/) (2025)
- [ZEUS: Accelerating Diffusion Models with Only Second-Order Predictor](https://mlsanity.com/paper/2604.01552/) (2026)
- [Not All Visual Tokens Are Equally Safe to Remove:Consequence-Sensitive Visual Token Compression](https://mlsanity.com/paper/2608.09176/) (2026)
- [Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models](https://mlsanity.com/paper/2608.21019/) (2026)
- [Joint Branch-Space Transform Coding for Diffusion Activation Quantization with Classifier-Free Guidance](https://mlsanity.com/paper/2610.00930/) (2026)

## Abstract (arXiv)

Diffusion language model (dLLM) compression faces a known challenge because calibration is typically performed on clean, fully visible activations, whereas inference traverses partially masked intermediate states. For low-rank compression, this raises two questions. First, can low-rank optimality still be characterized when approximation quality is measured over trajectory-distributed states, and second, does the choice of calibration states affect mathematical reasoning preservation under compression? We address these questions by formulating a trajectory-aware low-rank objective over corruption levels and masking realizations. To estimate this objective efficiently, we propose Traj-MC, which estimates the trajectory second moment through Monte Carlo sampling and yields exact sampled-state optimality and population consistency. Under matched compression budgets, trajectory-aware calibration improves reconstruction over the generation trajectory and preserves substantially more mathematical reasoning than clean calibration on mathematical reasoning benchmarks. Our results connect trajectory-aware low-rank optimality to the reasoning capability retained after dLLM compression. Our code is available at: https://github.com/Zishan-Shao/traj-mc.git.