# Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervision

arXiv: https://arxiv.org/abs/2206.14349 · HTML: https://mlsanity.com/paper/2206.14349/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 75 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 30 of 30)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [Masked Visual Pre-training for Motor Control](https://mlsanity.com/paper/2203.06173/) (2022)
- [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning](https://mlsanity.com/paper/1807.08364/) (2018)
- [Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions](https://mlsanity.com/paper/2203.15103/) (2022)
- [Query-Efficient Imitation Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/1605.06450/) (2016)
- [LazyDAgger: Reducing Context Switching in Interactive Imitation Learning](https://mlsanity.com/paper/2104.00053/) (2021)
- [Interactive Policy Learning through Confidence-Based Autonomy](https://mlsanity.com/paper/1401.3439/) (2014)
- [AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers](https://mlsanity.com/paper/1909.04121/) (2019)
- [LaND: Learning to Navigate from Disengagements](https://mlsanity.com/paper/2010.04689/) (2020)
- [Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning](https://mlsanity.com/paper/1812.05027/) (2018)
- [Scaled Autonomy: Enabling Human Operators to Control Robot Fleets](https://mlsanity.com/paper/1910.02910/) (2019)
- [Intervention Aided Reinforcement Learning for Safe and Practical Policy Optimization in Navigation](https://mlsanity.com/paper/1811.06187/) (2018)
- [Interactive Imitation Learning in State-Space](https://mlsanity.com/paper/2008.00524/) (2020)
- [Scalable Operator Allocation for Multi-Robot Assistance: A Restless Bandit Approach](https://mlsanity.com/paper/2111.06437/) (2021)
- [ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters](https://mlsanity.com/paper/2205.01906/) (2022)
- [Synergistic Scheduling of Learning and Allocation of Tasks in Human-Robot Teams](https://mlsanity.com/paper/2203.07478/) (2022)
- [Traversing Supervisor Problem: An Approximately Optimal Approach to Multi-Robot Assistance](https://mlsanity.com/paper/2205.01768/) (2022)

## Built on by (most cited first, 30 of 30)

- [Model-Based Runtime Monitoring with Interactive Imitation Learning](https://mlsanity.com/paper/2310.17552/) (2023)
- [Multi-Task Interactive Robot Fleet Learning with Visual World Models](https://mlsanity.com/paper/2410.22689/) (2024)
- [Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulation](https://mlsanity.com/paper/2410.14868/) (2024)
- [RLIF: Interactive Imitation Learning as Reinforcement Learning](https://mlsanity.com/paper/2311.12996/) (2023)
- [TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation](https://mlsanity.com/paper/2403.07869/) (2024)
- [Human-in-the-Loop Task and Motion Planning for Imitation Learning](https://mlsanity.com/paper/2310.16014/) (2023)
- [IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning](https://mlsanity.com/paper/2405.01472/) (2024)
- [SOP: A Scalable Online Post-Training System for Vision-Language-Action Models](https://mlsanity.com/paper/2601.03044/) (2026)
- [So You Think You Can Scale Up Autonomous Robot Data Collection?](https://mlsanity.com/paper/2411.01813/) (2024)
- [Human-assisted Robotic Policy Refinement via Action Preference Optimization](https://mlsanity.com/paper/2506.07127/) (2025)
- [Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based Reinforcement Learning](https://mlsanity.com/paper/2210.02343/) (2022)
- [Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback](https://mlsanity.com/paper/2410.08852/) (2024)
- [MILE: Model-based Intervention Learning](https://mlsanity.com/paper/2502.13519/) (2025)
- [Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Models](https://mlsanity.com/paper/2503.01876/) (2025)
- [Genie Centurion: Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidance](https://mlsanity.com/paper/2505.18793/) (2025)
- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)
- [Leveraging Demonstrator-perceived Precision for Safe Interactive Imitation Learning of Clearance-limited Tasks](https://mlsanity.com/paper/2402.13466/) (2024)
- [FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement](https://mlsanity.com/paper/2607.01111/) (2026)
- [IIFL: Implicit Interactive Fleet Learning from Heterogeneous Human Supervisors](https://mlsanity.com/paper/2306.15228/) (2023)
- [NeRF-Aug: Data Augmentation for Robotics with Neural Radiance Fields](https://mlsanity.com/paper/2411.02482/) (2024)
- [Robot See, Robot Do: Imitation Reward for Noisy Financial Environments](https://mlsanity.com/paper/2411.08637/) (2024)
- [R2BC: Multi-Agent Imitation Learning from Single-Agent Demonstrations](https://mlsanity.com/paper/2510.18085/) (2025)
- [Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning](https://mlsanity.com/paper/2511.14396/) (2025)
- [HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation Interface](https://mlsanity.com/paper/2609.20659/) (2026)
- [CubeDAgger: Interactive Imitation Learning for Dynamic Systems with Efficient yet Low-risk Interaction](https://mlsanity.com/paper/2505.04897/) (2025)
- [ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning](https://mlsanity.com/paper/2508.05310/) (2025)
- [Sample-Efficient Expert Query Control in Active Imitation Learning via Conformal Prediction](https://mlsanity.com/paper/2512.00453/) (2025)
- [Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via Assistant Experts](https://mlsanity.com/paper/2603.12769/) (2026)
- [Convex AI Compositionality and the Governance of AI System Populations](https://mlsanity.com/paper/2609.24784/) (2026)
- [Reward-DAgger: Robot-Gated Interactive Imitation Learning with General-Purpose Progress-Based Reward Models](https://mlsanity.com/paper/2610.04054/) (2026)

## Abstract (arXiv)

Commercial and industrial deployments of robot fleets at Amazon, Nimble, Plus One, Waymo, and Zoox query remote human teleoperators when robots are at risk or unable to make task progress. With continual learning, interventions from the remote pool of humans can also be used to improve the robot fleet control policy over time. A central question is how to effectively allocate limited human attention. Prior work addresses this in the single-robot, single-human setting; we formalize the Interactive Fleet Learning (IFL) setting, in which multiple robots interactively query and learn from multiple human supervisors. We propose Return on Human Effort (ROHE) as a new metric and Fleet-DAgger, a family of IFL algorithms. We present an open-source IFL benchmark suite of GPU-accelerated Isaac Gym environments for standardized evaluation and development of IFL algorithms. We compare a novel Fleet-DAgger algorithm to 4 baselines with 100 robots in simulation. We also perform a physical block-pushing experiment with 4 ABB YuMi robot arms and 2 remote humans. Experiments suggest that the allocation of humans to robots significantly affects the performance of the fleet, and that the novel Fleet-DAgger algorithm can achieve up to 8.8x higher ROHE than baselines. See https://tinyurl.com/fleet-dagger for supplemental material.