mlsanityData to 14 Oct 2026

Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning

arXiv 2608.22629 ↗First posted 23 Aug 2026cs.RO cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge, particularly when dealing with low-level torque control, where even small modelling inaccuracies can lead to unstable or unsafe behaviours. In this work, we propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms by combining trajectory matching, parameter optimization via genetic algorithms, and domain randomization. Using the 7-DOF Franka Emika Panda robot, we first identify friction, inertia, and gravity compensation parameters by minimizing the error between real and simulated joint trajectories. These calibrated dynamics are then used to train a TQC-based reinforcement learning agent in simulation. The trained policy is evaluated in both Gazebo and MuJoCo environments, and finally deployed on the real robot. Our results demonstrate a significant improvement in tracking accuracy and policy robustness after parameter tuning, with smooth policy transfer from simulation to the Real-World across multiple target-reaching tasks. This work highlights the effectiveness of accurate physical modelling in enabling stable and generalizable torque-based reinforcement learning policies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 23 references that are in the map. First posted on arXiv 23 Aug 2026.

Builds on

16 of 23 references.

References · 16

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  5. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  6. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  7. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 165
  8. 2021Auto-Tuned Sim-to-Real Transfercited by 49
  9. 2019Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assemblycited by 39
  10. 2023Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfercited by 22
  11. 2022DROPO: Sim-to-Real Transfer with Offline Domain Randomizationcited by 19
  12. 2020Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulationcited by 14
4 more
  1. 2022Selecting and Composing Learning Rate Policies for Deep Neural Networkscited by 12
  2. 2020A User's Guide to Calibrating Robotics Simulatorscited by 10
  3. 2023DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Controlcited by 7
  4. 2024Sim-to-Real gap in RL: Use Case with TIAGo and Isaac Sim/Gymcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
  2. 2025Towards bridging the gap: Systematic sim-to-real transfer for diverse legged robots
  3. 2026Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning ↗
  4. 2025Impact of Static Friction on Sim2Real in Robotic Reinforcement Learning
  5. 2024Improving generalization of robot locomotion policies via Sharpness-Aware Reinforcement Learning
  6. 2025Sim-to-Real Transfer for Mobile Robots with Reinforcement Learning: from NVIDIA Isaac Sim to Gazebo and Real ROS 2 Robots
  7. 2025Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation
  8. 2025Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion