mlsanityData to 14 Oct 2026

Facilitating Sim-to-real by Intrinsic Stochasticity of Real-Time Simulation in Reinforcement Learning for Robot Manipulation

arXiv 2304.06056 ↗First posted 12 Apr 2023cs.RO cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Simulation is essential to reinforcement learning (RL) before implementation in the real world, especially for safety-critical applications like robot manipulation. Conventionally, RL agents are sensitive to the discrepancies between the simulation and the real world, known as the sim-to-real gap. The application of domain randomization, a technique used to fill this gap, is limited to the imposition of heuristic-randomized models. {We investigate the properties of intrinsic stochasticity of real-time simulation (RT-IS) of off-the-shelf simulation software and its potential to improve RL performance. This improvement includes a higher tolerance to noise and model imprecision and superiority to conventional domain randomization in terms of ease of use and automation. Firstly, we conduct analytical studies to measure the correlation of RT-IS with the utilization of computer hardware and validate its comparability with the natural stochasticity of a physical robot. Then, we exploit the RT-IS feature in the training of an RL agent. The simulation and physical experiment results verify the feasibility and applicability of RT-IS to robust agent training for robot manipulation tasks. The RT-IS-powered RL agent outperforms conventional agents on robots with modeling uncertainties. RT-IS requires less heuristic randomization, is not task-dependent, and achieves better generalizability than the conventional domain-randomization-powered agents. Our findings provide a new perspective on the sim-to-real problem in practical applications like robot manipulation tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 63 references that are in the map. First posted on arXiv 12 Apr 2023.

Builds on

26 of 63 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2017Robust Adversarial Reinforcement Learningcited by 451
  8. 2017Parameter Space Noise for Explorationcited by 161
  9. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 144
  10. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  11. 2022Robust Reinforcement Learning using Offline Datacited by 96
  12. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 85
14 more
  1. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 83
  2. 2018Reinforcement Learning with Perturbed Rewardscited by 79
  3. 2020Reinforcement Learning with Random Delayscited by 59
  4. 2017Deep Learning in Robotics: A Review of Recent Researchcited by 43
  5. 2019Real-Time Reinforcement Learningcited by 40
  6. 2022Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robotcited by 38
  7. 2021SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learningcited by 27
  8. 2021Comparing Popular Simulation Environments in the Scope of Robotics and Reinforcement Learningcited by 12
  9. 2020A Comparison of Humanoid Robot Simulators: A Quantitative Approachcited by 8
  10. 2020Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboardcited by 7
  11. 2022ParaPose: Parameter and Domain Randomization Optimization for Pose Estimation using Synthetic Datacited by 5
  12. 2022Deep transfer learning for system identification using long short-term memory neural networkscited by 4
  13. 2022Domain Randomization for Object Countingcited by 2
  14. 2018Nonlinear System Identification using Neural Networks and Trajectory-Based Optimizationcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasetscited by 9
  2. 2024Meta SAC-Lag: Towards Deployable Safe Reinforcement Learning via MetaGradient-based Hyperparameter Tuningcited by 3