# Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation

arXiv: https://arxiv.org/abs/2607.14557 · HTML: https://mlsanity.com/paper/2607.14557/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning.
- Builds on 42 of its 68 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Placed in

- LLM reasoning / [Diffusion reasoning and test-time scaling](https://mlsanity.com/lr/diffusion-reasoning-and-test-time-scaling/)
- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 30 of 42)

- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025)
- [dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching](https://mlsanity.com/paper/2506.06295/) (2025)
- [Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs](https://mlsanity.com/paper/2412.01818/) (2024)
- [Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More](https://mlsanity.com/paper/2502.11494/) (2025)
- [Accelerating Diffusion LLMs via Adaptive Parallel Decoding](https://mlsanity.com/paper/2506.00413/) (2025)
- [Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding](https://mlsanity.com/paper/2505.16990/) (2025)
- [ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models](https://mlsanity.com/paper/2412.00447/) (2024)
- [LaViDa: A Large Diffusion Language Model for Multimodal Understanding](https://mlsanity.com/paper/2505.16839/) (2025)
- [FastVLM: Efficient Vision Encoding for Vision Language Models](https://mlsanity.com/paper/2412.13303/) (2024)
- [Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction](https://mlsanity.com/paper/2508.02558/) (2025)
- [Any-Order Flexible Length Masked Diffusion](https://mlsanity.com/paper/2509.01025/) (2025)
- [PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models](https://mlsanity.com/paper/2504.08966/) (2025)
- [Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention](https://mlsanity.com/paper/2510.02912/) (2025)
- [AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning](https://mlsanity.com/paper/2412.03248/) (2024)
- [DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding](https://mlsanity.com/paper/2510.02358/) (2025)
- [What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graph](https://mlsanity.com/paper/2501.02268/) (2025)
- [MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs](https://mlsanity.com/paper/2508.18264/) (2025)
- [Attention Is All You Need for KV Cache in Diffusion LLMs](https://mlsanity.com/paper/2510.14973/) (2025)
- [Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction](https://mlsanity.com/paper/2412.00556/) (2024)
- [DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference](https://mlsanity.com/paper/2601.19278/) (2026)

## Built on by (most cited first, 5 of 5)

- [Geometry-guided Emotion Modulation for Controllable and Photorealistic Emotional Talking Face Generation](https://mlsanity.com/paper/2608.00663/) (2026)
- [Understanding and Overcoming Cross-modal Fusion Bias in Multimodal Anomaly Detection From A Fisher Information Perspective](https://mlsanity.com/paper/2608.00986/) (2026)
- [LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search](https://mlsanity.com/paper/2608.09152/) (2026)
- [Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering](https://mlsanity.com/paper/2608.21450/) (2026)
- [MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agents](https://mlsanity.com/paper/2609.19059/) (2026)

## Abstract (arXiv)

Diffusion Multimodal Large Language Models (DMLLMs) are highly effective for multimodal reasoning, yet their inference efficiency is significantly hindered by fixed-length generation constraints. Since the actual output length is unknown, output sequences are padded to a predefined maximum length, resulting in substantial redundant computation over unnecessary [EOS] tokens. In this work, we discover that DMLLMs implicitly reveal their valid semantic boundary at the very first denoising step through a distinct shift in MLP activation sparsity. Leveraging this observation, we propose Seer, a training-free framework that detects this boundary using a Signal-to-Noise Ratio (SNR)-based criterion and performs one-shot truncation of the redundant suffix for all subsequent computations. To preserve these theoretical gains during batched serving, Seer incorporates a hybrid execution strategy that maximizes throughput while seamlessly accommodating dynamic sequence lengths. Experimental results demonstrate that Seer effectively eliminates padding waste, accelerating throughput by up to ~31×. Across 9 benchmarks, Seer robustly maintains overall performance and even improves accuracy on complex visual tasks by mitigating noise leakage (e.g., DocVQA score increases from 63.52 to 63.66), offering a highly efficient, plug-and-play solution for DMLLM acceleration.