# VADF: Vision-Adaptive Diffusion Policy Framework for Efficient Robotic Manipulation

arXiv: https://arxiv.org/abs/2604.15938 · HTML: https://mlsanity.com/paper/2604.15938/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 29 of its 33 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models](https://mlsanity.com/paper/2503.22020/) (2025)
- [Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning](https://mlsanity.com/paper/1910.11956/) (2019)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [RoboDreamer: Learning Compositional World Models for Robot Imagination](https://mlsanity.com/paper/2404.12377/) (2024)
- [Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation](https://mlsanity.com/paper/2405.07503/) (2024)
- [Learning to Act from Actionless Videos through Dense Correspondences](https://mlsanity.com/paper/2310.08576/) (2023)
- [HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation](https://mlsanity.com/paper/2502.05485/) (2025)
- [One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation](https://mlsanity.com/paper/2410.21257/) (2024)
- [InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy](https://mlsanity.com/paper/2510.13778/) (2025)
- [HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers](https://mlsanity.com/paper/2410.05273/) (2024)
- [Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learning](https://mlsanity.com/paper/2402.17768/) (2024)
- [ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning](https://mlsanity.com/paper/2503.23297/) (2025)
- [Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysis](https://mlsanity.com/paper/2407.12173/) (2024)
- [ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation](https://mlsanity.com/paper/2601.08325/) (2026)
- [RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow Trajectories](https://mlsanity.com/paper/2503.07699/) (2025)
- [Adaptive Non-uniform Timestep Sampling for Accelerating Diffusion Model Training](https://mlsanity.com/paper/2411.09998/) (2024)
- [Spatial-Temporal Aware Visuomotor Diffusion Policy Learning](https://mlsanity.com/paper/2507.06710/) (2025)
- [Two-Steps Diffusion Policy for Robotic Manipulation via Genetic Denoising](https://mlsanity.com/paper/2510.21991/) (2025)
- [ALDM-Grasping: Diffusion-aided Zero-Shot Sim-to-Real Transfer for Robot Grasping](https://mlsanity.com/paper/2403.11459/) (2024)
- [Optimal Stepsize for Diffusion Sampling](https://mlsanity.com/paper/2503.21774/) (2025)
- [A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding](https://mlsanity.com/paper/2507.06719/) (2025)
- [Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling](https://mlsanity.com/paper/2510.23285/) (2025)
- [D2PPO: Diffusion Policy Policy Optimization with Dispersive Loss](https://mlsanity.com/paper/2508.02644/) (2025)
- [DexDiff: Towards Extrinsic Dexterity Manipulation of Ungraspable Objects in Unrestricted Environments](https://mlsanity.com/paper/2409.05493/) (2024)

## Abstract (arXiv)

Diffusion policies are becoming mainstream in robotic manipulation but suffer from hard negative class imbalance due to uniform sampling and lack of sample difficulty awareness, leading to slow training convergence and frequent inference timeout failures. We propose VADF (Vision-Adaptive Diffusion Policy Framework), a vision-driven dual-adaptive framework that significantly reduces convergence steps and achieves early success in inference, with model-agnostic design enabling seamless integration into any diffusion policy architecture. During training, we introduce Adaptive Loss Network (ALN), a lightweight MLP-based loss predictor that quantifies per-step sample difficulty in real time. Guided by hard negative mining, it performs weighted sampling to prioritize high-loss regions, enabling adaptive weight updates and faster convergence. In inference, we design the Hierarchical Vision Task Segmenter (HVTS), which decomposes high-level task instructions into multi-stage low-level sub-instructions based on visual input. It adaptively segments action sequences into simple and complex subtasks by assigning shorter noise schedules with longer direct execution sequences to simple actions, and longer noise steps with shorter execution sequences to complex ones, thereby dramatically reducing computational overhead and significantly improving the early success rate.