mlsanityData to 7 Oct 2026

Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment

arXiv 2211.08416 ↗First posted 15 Nov 2022cs.RO cs.AI cs.LGcited by 101

Not on the map. Why?

Cited by 29 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

With the rapid growth of computing powers and recent advances in deep learning, we have witnessed impressive demonstrations of novel robot capabilities in research settings. Nonetheless, these learning systems exhibit brittle generalization and require excessive training data for practical tasks. To harness the capabilities of state-of-the-art robot learning models while embracing their imperfections, we present Sirius, a principled framework for humans and robots to collaborate through a division of work. In this framework, partially autonomous robots are tasked with handling a major portion of decision-making where they work reliably; meanwhile, human operators monitor the process and intervene in challenging situations. Such a human-robot team ensures safe deployments in complex tasks. Further, we introduce a new learning algorithm to improve the policy's performance on the data collected from the task executions. The core idea is re-weighing training samples with approximated human trust and optimizing the policies with weighted behavioral cloning. We evaluate Sirius in simulation and on real hardware, showing that Sirius consistently outperforms baselines over a collection of contact-rich manipulation tasks, achieving an 8% boost in simulation and 27% on real hardware than the state-of-the-art methods in policy success rate, with twice faster convergence and 85% memory size reduction. Videos and more details are available at https://ut-austin-rpl.github.io/sirius/

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 51 of its 77 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

51 of 77 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  9. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  10. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  11. 2018Learning Dexterous In-Hand Manipulationcited by 643
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
38 more
  1. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  2. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  3. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  4. 2021Implicit Behavioral Cloningcited by 442
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  7. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  8. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  9. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  10. 2020Critic Regularized Regressioncited by 269
  11. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  12. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  13. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  14. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  15. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  16. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 122
  17. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrationscited by 114
  18. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  19. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 98
  20. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 96
  21. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  22. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 86
  23. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  24. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  25. 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
  26. 2020Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferencescited by 71
  27. 2018Shared Autonomy via Deep Reinforcement Learningcited by 65
  28. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  29. 2022Interactive Imitation Learning in Robotics: A Surveycited by 51
  30. 2015Shared Autonomy via Hindsight Optimizationcited by 46
  31. 2019Leveraging Human Guidance for Deep Reinforcement Learning Taskscited by 42
  32. 2021A Survey on Interactive Reinforcement Learning: Design Principles and Open Challengescited by 41
  33. 2021Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedbackcited by 34
  34. 2021Correct Me if I am Wrong: Interactive Learning for Robotic Manipulationcited by 24
  35. 2017Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practicescited by 22
  36. 2022Eliciting Compatible Demonstrations for Multi-Human Imitation Learningcited by 22
  37. 2022Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Visioncited by 11
  38. 2022Modeling Human Response to Robot Errors for Timely Error Detectioncited by 7

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 23; 2025: 42; 2026: 32.

101 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  3. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  4. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  5. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  6. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  7. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  8. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  9. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  10. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  11. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 123
  12. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
38 more
  1. 2024IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AIcited by 68
  2. 2025villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Modelscited by 58
  3. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  4. 2025Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructionscited by 50
  5. 2024RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learningcited by 46
  6. 2024RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulationcited by 40
  7. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  8. 2025RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correctioncited by 35
  9. 2024Multi-Task Interactive Robot Fleet Learning with Visual World Modelscited by 33
  10. 2023Interactive Robot Learning from Verbal Correctioncited by 28
  11. 2025Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Correctionscited by 25
  12. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learningcited by 24
  13. 2024Bimanual Dexterity for Complex Taskscited by 22
  14. 2025MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulationcited by 22
  15. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 20
  16. 2025What Matters in Learning from Large-Scale Datasets for Robot Manipulationcited by 20
  17. 2024So You Think You Can Scale Up Autonomous Robot Data Collection?cited by 17
  18. 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 17
  19. 2025GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulationcited by 16
  20. 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representationscited by 15
  21. 2025What Matters for Batch Online Reinforcement Learning in Robotics?cited by 14
  22. 2025Human-assisted Robotic Policy Refinement via Action Preference Optimizationcited by 13
  23. 2025How to Train Your Robots? The Impact of Demonstration Modality on Imitation Learningcited by 11
  24. 2024Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisitioncited by 10
  25. 2025From Screens to Scenes: A Survey of Embodied AI in Healthcarecited by 10
  26. 2025Guiding Data Collection via Factored Scaling Curvescited by 10
  27. 2025TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Modelscited by 10
  28. 2025HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policiescited by 10
  29. 2026MMaDA-VLA: Large Diffusion Vision-Language-Action Model with Unified Multi-Modal Instruction and Generationcited by 10
  30. 2025SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learningcited by 9
  31. 2025Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretrainingcited by 8
  32. 2025MILE: Model-based Intervention Learningcited by 7
  33. 2025Multi-Agent LLM Actor-Critic Framework for Social Robot Navigationcited by 7
  34. 2025LatBot: Distilling Universal Latent Actions for Vision-Language-Action Modelscited by 7
  35. 2026RotVLA: Rotational Latent Action for Vision-Language-Action Modelcited by 7
  36. 2025Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Modelscited by 6
  37. 2025SIME: Enhancing Policy Self-Improvement with Modal-level Explorationcited by 6
  38. 2025Quantum Verifiable Rewards for Post-Training Qiskit Code Assistantcited by 6

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026HARBOR: A Harness Framework for Agentic Robot Reinforcement Learning
  2. 2025Robot Learning: A Tutorial
  3. 2024ARMCHAIR: integrated inverse reinforcement learning and model predictive control for human-robot collaboration
  4. 2026Nautilus: From One Prompt to Plug-and-Play Robot Learning
  5. 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies
  6. 2025Control Your Robot: A Unified System for Robot Control and Policy Deployment ↗
  7. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
  8. 2026AXIS: A Growable Community-Driven Data Engine for Scalable Robot Manipulation