# Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulation

arXiv: https://arxiv.org/abs/2410.14868 · HTML: https://mlsanity.com/paper/2410.14868/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Your Diffusion Model is Secretly a Zero-Shot Classifier](https://mlsanity.com/paper/2303.16203/) (2023)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI](https://mlsanity.com/paper/2410.00425/) (2024)
- [ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations](https://mlsanity.com/paper/2107.14483/) (2021)
- [Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models](https://mlsanity.com/paper/2308.01557/) (2023)
- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning](https://mlsanity.com/paper/1807.08364/) (2018)
- [Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation](https://mlsanity.com/paper/2112.05251/) (2021)
- [Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning](https://mlsanity.com/paper/2307.04726/) (2023)
- [Model-Based Runtime Monitoring with Interactive Imitation Learning](https://mlsanity.com/paper/2310.17552/) (2023)
- [Adaptive Online Replanning with Diffusion Models](https://mlsanity.com/paper/2310.09629/) (2023)
- [Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision](https://mlsanity.com/paper/2206.14349/) (2022)
- [PATO: Policy Assisted TeleOperation for Scalable Robot Data Collection](https://mlsanity.com/paper/2212.04708/) (2022)
- [DiffAIL: Diffusion Adversarial Imitation Learning](https://mlsanity.com/paper/2312.06348/) (2023)

## Built on by (most cited first, 30 of 33)

- [Failure Prediction at Runtime for Generative Robot Policies](https://mlsanity.com/paper/2510.09459/) (2025)
- [When to Trust Imagination: Adaptive Action Execution for World Action Models](https://mlsanity.com/paper/2605.06222/) (2026)
- [Demystifying Diffusion Policies: Action Memorization and Simple Lookup Table Alternatives](https://mlsanity.com/paper/2505.05787/) (2025)
- [Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Models](https://mlsanity.com/paper/2503.01876/) (2025)
- [Dynamic Rank Adjustment in Diffusion Policies for Efficient and Flexible Training](https://mlsanity.com/paper/2502.03822/) (2025)
- [FlowCorrect: Efficient Interactive Correction of Generative Flow Policies for Robotic Manipulation](https://mlsanity.com/paper/2602.22056/) (2026)
- [Accelerating Residual Reinforcement Learning with Uncertainty Estimation](https://mlsanity.com/paper/2506.17564/) (2025)
- [Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Survey](https://mlsanity.com/paper/2506.00098/) (2025)
- [DAgger Diffusion Navigation: DAgger Boosted Diffusion Policy for Vision-Language Navigation](https://mlsanity.com/paper/2508.09444/) (2025)
- [Uncertainty Quantification for Flow-Based Vision-Language-Action Models](https://mlsanity.com/paper/2606.18043/) (2026)
- [From Action Labels to Sets: Rethinking Action Supervision for Imitation Learning from Corrective Feedback](https://mlsanity.com/paper/2502.07645/) (2025)
- [Generative Control as Optimization: Time Unconditional Flow Matching for Adaptive and Robust Robotic Control](https://mlsanity.com/paper/2603.17834/) (2026)
- [Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness](https://mlsanity.com/paper/2603.21017/) (2026)
- [ActProbe: Action-Space Probe for Early Failure Detection of Generative Robot Policies](https://mlsanity.com/paper/2606.08508/) (2026)
- [VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models](https://mlsanity.com/paper/2606.21386/) (2026)
- [RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models](https://mlsanity.com/paper/2606.23617/) (2026)
- [PANDORA: Diffusion Policy Learning for Dexterous Robotic Piano Playing](https://mlsanity.com/paper/2503.14545/) (2025)
- [Hand-in-the-Loop: Improving VLA Policies for Dexterous Manipulation via Seamless Hand-Arm Intervention](https://mlsanity.com/paper/2605.15157/) (2026)
- [The Geometry of Flow-Matching Uncertainty: A Cost-free Uncertainty Proxy and Its Application in Flow-based VLA Failure Detection](https://mlsanity.com/paper/2607.27933/) (2026)
- [Selective Progress-Aware Querying for Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2509.20541/) (2025)
- [Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy](https://mlsanity.com/paper/2511.00555/) (2025)
- [Set-Supervised Diffusion Policy: Learning Action-Chunking Diffusion through Corrections](https://mlsanity.com/paper/2606.01865/) (2026)
- [DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning](https://mlsanity.com/paper/2609.08123/) (2026)
- [ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning](https://mlsanity.com/paper/2508.05310/) (2025)
- [VoxelDiffusionCut: Non-destructive Internal-part Extraction via Iterative Cutting and Structure Estimation](https://mlsanity.com/paper/2603.00116/) (2026)
- [Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models](https://mlsanity.com/paper/2603.05147/) (2026)
- [Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via Assistant Experts](https://mlsanity.com/paper/2603.12769/) (2026)
- [VE2VF: Vision-Enabled to Vision-Free Distillation via Real-world Reinforcement Learning for Robust Contact-Rich Manipulation](https://mlsanity.com/paper/2605.29564/) (2026)
- [ReflectVLN: Training Vision-Language Navigation Agents with Reflective Reasoning](https://mlsanity.com/paper/2607.12680/) (2026)
- [Making two action heads agree: coordination mechanisms and a runtime collapse certificate for flow-matching policies](https://mlsanity.com/paper/2608.15748/) (2026)

## Abstract (arXiv)

Recently, diffusion policy has shown impressive results in handling multi-modal tasks in robotic manipulation. However, it has fundamental limitations in out-of-distribution failures that persist due to compounding errors and its limited capability to extrapolate. One way to address these limitations is robot-gated DAgger, an interactive imitation learning with a robot query system to actively seek expert help during policy rollout. While robot-gated DAgger has high potential for learning at scale, existing methods like Ensemble-DAgger struggle with highly expressive policies: They often misinterpret policy disagreements as uncertainty at multi-modal decision points. To address this problem, we introduce Diff-DAgger, an efficient robot-gated DAgger algorithm that leverages the training objective of diffusion policy. We evaluate Diff-DAgger across different robot tasks including stacking, pushing, and plugging, and show that Diff-DAgger improves the task failure prediction by 39.0%, the task completion rate by 20.6%, and reduces the wall-clock time by a factor of 7.8. We hope that this work opens up a path for efficiently incorporating expressive yet data-hungry policies into interactive robot learning settings. The project website is available at: https://diffdagger.github.io.