# Block-wise Adaptive Caching for Accelerating Diffusion Policy

arXiv: https://arxiv.org/abs/2506.13456 · HTML: https://mlsanity.com/paper/2506.13456/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 3 of its 4 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 3 of 3)

- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model](https://mlsanity.com/paper/2411.19108/) (2024)
- [Diffusion Transformer Policy](https://mlsanity.com/paper/2410.15959/) (2024)

## Built on by (most cited first, 12 of 12)

- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion](https://mlsanity.com/paper/2508.21091/) (2025)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment](https://mlsanity.com/paper/2604.24447/) (2026)
- [TS-DP: Reinforcement Speculative Decoding For Temporal Adaptive Diffusion Policy Acceleration](https://mlsanity.com/paper/2512.15773/) (2025)
- [Sparse ActionGen: Accelerating Diffusion Policy with Real-time Pruning](https://mlsanity.com/paper/2601.12894/) (2026)
- [No Cache Left Idle: Accelerating diffusion model via Extreme-slimming Caching](https://mlsanity.com/paper/2512.12604/) (2025)
- [ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models](https://mlsanity.com/paper/2605.29438/) (2026)
- [Evolving Cache Schedules for Fast Diffusion Policy Inference](https://mlsanity.com/paper/2607.20293/) (2026)
- [Test-time Sparsity for Extreme Fast Action Diffusion](https://mlsanity.com/paper/2605.13316/) (2026)
- [Urgent Actions Go First: Urgency-Aware Denoising for Real-Time VLA Control](https://mlsanity.com/paper/2609.37772/) (2026)
- [MARS: Systematic Multi-Level Acceleration for World Action Models](https://mlsanity.com/paper/2610.17141/) (2026)

## Abstract (arXiv)

Diffusion Policy has demonstrated strong visuomotor modeling capabilities, but its high computational cost renders it impractical for real-time robotic control. Despite huge redundancy across repetitive denoising steps, existing diffusion acceleration techniques fail to generalize to Diffusion Policy due to fundamental architectural and data divergences. In this paper, we propose Block-wise Adaptive Caching (BAC), a method to accelerate Diffusion Policy by caching intermediate action features. BAC achieves lossless action generation acceleration by adaptively updating and reusing cached features at the block level, based on a key observation that feature similarities exhibit non-uniform temporal dynamics and distinct block-specific patterns. To operationalize this insight, we first design an Adaptive Caching Scheduler to identify optimal update timesteps by maximizing the global feature similarities between cached and skipped features. However, applying this scheduler for each block leads to significant error surges due to the inter-block propagation of caching errors, particularly within Feed-Forward Network (FFN) blocks. To mitigate this issue, we develop the Bubbling Union Algorithm, which truncates these errors by updating the upstream blocks with significant caching errors before downstream FFNs. As a training-free plugin, BAC is readily integrable with existing transformer-based Diffusion Policy and vision-language-action models. Extensive experiments on multiple robotic benchmarks demonstrate that BAC achieves up to 3× inference speedup for free. Project page: https://block-wise-adaptive-caching.github.io.