mlsanityData to 7 Oct 2026

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction

arXiv 2405.10315 ↗First posted 16 May 2024cs.RO cs.AI cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

Learning in simulation and transferring the learned policy to the real world has the potential to enable generalist robots. The key challenge of this approach is to address simulation-to-reality (sim-to-real) gaps. Previous methods often require domain-specific knowledge a priori. We argue that a straightforward way to obtain such knowledge is by asking humans to observe and assist robot policy execution in the real world. The robots can then learn from humans to close various sim-to-real gaps. We propose TRANSIC, a data-driven approach to enable successful sim-to-real transfer based on a human-in-the-loop framework. TRANSIC allows humans to augment simulation policies to overcome various unmodeled sim-to-real gaps holistically through intervention and online correction. Residual policies can be learned from human corrections and integrated with simulation policies for autonomous execution. We show that our approach can achieve successful sim-to-real transfer in complex and contact-rich manipulation tasks such as furniture assembly. Through synergistic integration of policies learned in simulation and from humans, TRANSIC is effective as a holistic approach to addressing various, often coexisting sim-to-real gaps. It displays attractive properties such as scaling with human effort. Videos and code are available at https://transic-robot.github.io/

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 119 of its 172 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

119 of 172 references.

References · 119

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  12. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
38 more
  1. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  3. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  4. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  5. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  6. 2019Solving Rubik's Cube with a Robot Handcited by 699
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  8. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  9. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  10. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  11. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  12. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  13. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  14. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  15. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  16. 2015Policy Distillationcited by 414
  17. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  18. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  19. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  20. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  21. 2018Maximum a Posteriori Policy Optimisationcited by 318
  22. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  23. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  24. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  25. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  26. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  27. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  28. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  29. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 180
  30. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 171
  31. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  32. 2021A System for General In-Hand Object Re-Orientationcited by 155
  33. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  34. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  35. 2018Residual Policy Learningcited by 146
  36. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 138
  37. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  38. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135

50 most cited of 119 All 119 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 17; 2026: 20.

50 papers.

Sort:

By area:

  1. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  2. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  3. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
  4. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  5. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
  6. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  7. 2024Hand-Object Interaction Pretraining from Videoscited by 38
  8. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 36
  9. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  10. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  11. 2024Lessons from Learning to Spin "Pens"cited by 22
  12. 2024Local Policies Enable Zero-shot Long-horizon Manipulationcited by 21
38 more
  1. 2024Neural MP: A Generalist Neural Motion Plannercited by 18
  2. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfercited by 13
  3. 2025Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twincited by 11
  4. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  5. 2025VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuningcited by 10
  6. 2024Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Graspingcited by 8
  7. 2025MILE: Model-based Intervention Learningcited by 7
  8. 2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7
  9. 2025SIME: Enhancing Policy Self-Improvement with Modal-level Explorationcited by 6
  10. 2026FlowCorrect: Efficient Interactive Correction of Generative Flow Policies for Robotic Manipulationcited by 5
  11. 2025A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewardscited by 4
  12. 2026When to Act, Ask, or Learn: Uncertainty-Aware Policy Steeringcited by 4
  13. 2026UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learningcited by 4
  14. 2026What Makes Value Learning Efficient in Residual Reinforcement Learning?cited by 3
  15. 2026FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Spacecited by 3
  16. 2025Factorizing Diffusion Policies for Observation Modality Prioritizationcited by 2
  17. 2024MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Interventioncited by 1
  18. 2025FoldNet: Learning Generalizable Closed-Loop Policy for Garment Folding via Keypoint-Driven Asset and Demonstration Synthesiscited by 1
  19. 2026StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perceptioncited by 1
  20. 2026Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancementcited by 1
  21. 2026Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?cited by 1
  22. 2026DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Modelcited by 1
  23. 2026HarnessPAI: An Evolving Harness for Physical AIcited by 1
  24. 2025Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardwarecited by 0
  25. 2025Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrationscited by 0
  26. 2025Query-Centric Diffusion Policy for Generalizable Robotic Assemblycited by 0
  27. 2025Data-Efficient Multitask DAggercited by 0
  28. 2025Fine-Tuning Flow Matching via Maximum Likelihood Estimation of Reconstructionscited by 0
  29. 2026TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulationcited by 0
  30. 2026HiL-ResRL: A Model-Agnostic Finetuning Adapter via Human-in-the-loop Residual Reinforcement Learningcited by 0
  31. 2026SILO: Simulation-in-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routingcited by 0
  32. 2026Shared Voxel-Map-Based Cooperative Indoor UAV Guidance with a Multi-Agent Soft Actor-Critic Controllercited by 0
  33. 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0
  34. 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Controlcited by 0
  35. 2026HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation Interfacecited by 0
  36. 2026Dissecting Advantage-Guided Post-Training for Vision-Language-Action Policiescited by 0
  37. 2026VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RLcited by 0
  38. 2026BlenDAgger: Blended Shared Control for Interactive Imitation Learningcited by 0