mlsanityData to 14 Oct 2026

Digital Twin-Driven Reinforcement Learning for Obstacle Avoidance in Robot Manipulators: A Self-Improving Online Training Framework

arXiv 2403.13090 ↗First posted 19 Mar 2024cs.RO cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

The evolution and growing automation of collaborative robots introduce more complexity and unpredictability to systems, highlighting the crucial need for robot's adaptability and flexibility to address the increasing complexities of their environment. In typical industrial production scenarios, robots are often required to be re-programmed when facing a more demanding task or even a few changes in workspace conditions. To increase productivity, efficiency and reduce human effort in the design process, this paper explores the potential of using digital twin combined with Reinforcement Learning (RL) to enable robots to generate self-improving collision-free trajectories in real time. The digital twin, acting as a virtual counterpart of the physical system, serves as a 'forward run' for monitoring, controlling, and optimizing the physical system in a safe and cost-effective manner. The physical system sends data to synchronize the digital system through the video feeds from cameras, which allows the virtual robot to update its observation and policy based on real scenarios. The bidirectional communication between digital and physical systems provides a promising platform for hardware-in-the-loop RL training through trial and error until the robot successfully adapts to its new environment. The proposed online training framework is demonstrated on the Unfactory Xarm5 collaborative robot, where the robot end-effector aims to reach the target position while avoiding obstacles. The experiment suggest that proposed framework is capable of performing policy online training, and that there remains significant room for improvement.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 35 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

6 of 35 references.

References · 6

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2019Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assemblycited by 39
  4. 2021Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Studycited by 30
  5. 2020Man, machine and work in a digital twin setup: a case studycited by 2
  6. 2022A Systematic Comparison of Simulation Software for Robotic Arm Manipulation using ROS2cited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility
  2. 2024A Digital Twin Framework for Reinforcement Learning with Real-Time Self-Improvement via Human Assistive Teleoperation
  3. 2026TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulation
  4. 2025Digital Twin Synchronization: Bridging the Sim-RL Agent to a Real-Time Robotic Additive Manufacturing Control
  5. 2025Hybrid Robot Learning for Automatic Robot Motion Planning in Manufacturing
  6. 2025URPlanner: A Universal Paradigm For Collision-Free Robotic Motion Planning Based on Deep Reinforcement Learning
  7. 2024Collaborative motion planning for multi-manipulator systems through Reinforcement Learning and Dynamic Movement Primitives
  8. 2022Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin