mlsanityData to 7 Oct 2026

Residual Reinforcement Learning for Robot Control

arXiv 1812.03201 ↗First posted 7 Dec 2018cs.RO cs.LGcited by 242

Not on the map. Why?

Cited by 31 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Conventional feedback control methods can solve various types of robot control problems very efficiently by capturing the structure with explicit models, such as rigid body equations of motion. However, many control problems in modern manufacturing deal with contacts and friction, which are difficult to capture with first-order physical modeling. Hence, applying control design methodologies to these kinds of problems often results in brittle and inaccurate controllers, which have to be manually tuned for deployment. Reinforcement learning (RL) methods have been demonstrated to be capable of learning continuous robot controllers from interactions with the environment, even for problems that include friction and contacts. In this paper, we study how we can solve difficult control problems in the real world by decomposing them into a part that is solved efficiently by conventional feedback control methods, and the residual which is solved with RL. The final control policy is a superposition of both control signals. We demonstrate our approach by training an agent to successfully perform a real-world block assembly task involving contacts and unstable objects.

Key facts. Cited by 242 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 28; 2022: 24; 2023: 24; 2024: 35; 2025: 30; 2026: 99.

242 papers.

Sort:

By area:

  1. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  2. 2022Human-to-Robot Imitation in the Wildcited by 157
  3. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  4. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  5. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  6. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
  7. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 95
  8. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  9. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  10. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  11. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  12. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
38 more
  1. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  2. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  3. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  4. 2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learningcited by 42
  5. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  6. 2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 39
  7. 2023Continual Semantic Segmentation with Automatic Memory Sample Selectioncited by 39
  8. 2021Offline Meta-Reinforcement Learning for Industrial Insertioncited by 37
  9. 2022Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learningcited by 36
  10. 2024Grounding Multimodal Large Language Models in Actionscited by 35
  11. 2025ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learningcited by 34
  12. 2023Human-in-the-Loop Task and Motion Planning for Imitation Learningcited by 28
  13. 2025DiWA: Diffusion Policy Adaptation with World Modelscited by 28
  14. 2021Residual Reinforcement Learning from Demonstrationscited by 27
  15. 2025Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Correctionscited by 25
  16. 2021Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Studycited by 24
  17. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 23
  18. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 23
  19. 2024Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimizationcited by 23
  20. 2020Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulationcited by 22
  21. 2022Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learningcited by 22
  22. 2023A Survey on Physics Informed Reinforcement Learning: Review and Open Problemscited by 22
  23. 2024Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusioncited by 21
  24. 2022Robot Learning of Mobile Manipulation with Reachability Behavior Priorscited by 20
  25. 2025Fast Adaptation with Behavioral Foundation Modelscited by 20
  26. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 18
  27. 2021Imitation Learning for Variable Speed Contact Motion for Operation up to Control Bandwidthcited by 16
  28. 2021Composable Energy Policies for Reactive Motion Generation and Reinforcement Learningcited by 16
  29. 2022SafeLight: A Reinforcement Learning Method toward Collision-free Traffic Signal Controlcited by 15
  30. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 15
  31. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priorscited by 15
  32. 2025Touch begins where vision ends: Generalizable policies for contact-rich manipulationcited by 13
  33. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfercited by 13
  34. 2022Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecastingcited by 12
  35. 2021Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policiescited by 11
  36. 2022Human-Centric Artificial Intelligence Architecture for Industry 5.0 Applicationscited by 11
  37. 2022ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representationcited by 11
  38. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Modelscited by 11

50 most cited of 242 All 242 (JSON)