mlsanityData to 14 Oct 2026

Provable Sim-to-real Transfer in Continuous Domain with Partial Observations

arXiv 2210.15598 ↗First posted 27 Oct 2022cs.LG cs.AI stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Sim-to-real transfer trains RL agents in the simulated environments and then deploys them in the real world. Sim-to-real transfer has been widely used in practice because it is often cheaper, safer and much faster to collect samples in simulation than in the real world. Despite the empirical success of the sim-to-real transfer, its theoretical foundation is much less understood. In this paper, we study the sim-to-real transfer in continuous domain with partial observations, where the simulated environments and real-world environments are modeled by linear quadratic Gaussian (LQG) systems. We show that a popular robust adversarial training algorithm is capable of learning a policy from the simulated environment that is competitive to the optimal policy in the real-world environment. To achieve our results, we design a new algorithm for infinite-horizon average-cost LQGs and establish a regret bound that depends on the intrinsic complexity of the model class. Our algorithm crucially relies on a novel history clipping scheme, which might be of independent interest.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 34 of its 70 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

34 of 70 references.

References · 34

  1. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  2. 2018Learning Dexterous In-Hand Manipulationcited by 687
  3. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  4. 2017Robust Adversarial Reinforcement Learningcited by 451
  5. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  6. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  7. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  8. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 234
  9. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  10. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 223
  11. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  12. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 154
22 more
  1. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  2. 2020Naive Exploration is Optimal for Online LQRcited by 116
  3. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  4. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 94
  5. 2020Improper Learning for Non-Stochastic Controlcited by 83
  6. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 80
  7. 2018Non-asymptotic Identification of LTI Systems from a Single Trajectorycited by 78
  8. 2019Provably Efficient Q-Learning with Low Switching Costcited by 78
  9. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 78
  10. 2018Online Linear Quadratic Controlcited by 56
  11. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 54
  12. 2020Logarithmic Regret Bound in Partially Observable Linear Dynamical Systemscited by 46
  13. 2020Black-Box Control for Linear Dynamical Systemscited by 43
  14. 2021On the Regret Analysis of Online LQR Control with Predictionscited by 22
  15. 2020Reinforcement Learning with Fast Stabilization in Linear Dynamical Systemscited by 20
  16. 2021Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimizationcited by 18
  17. 2016Supervision via Competition: Robot Adversaries for Learning Taskscited by 16
  18. 2020Sample Complexity of Linear Quadratic Gaussian (LQG) Control for Output Feedback Systemscited by 16
  19. 2019Certainty Equivalence is Efficient for Linear Quadratic Controlcited by 13
  20. 2019Sample Complexity of Kalman Filtering for Unknown Systemscited by 12
  21. 2020Regret Minimization in Partially Observable Linear Quadratic Controlcited by 7
  22. 2019PAC Reinforcement Learning without Real-World Feedbackcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4; 2025: 3.

9 papers.

  1. 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
  2. 2024Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithmscited by 26
  3. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
  4. 2025Generative Models in Decision Making: A Surveycited by 15
  5. 2024Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximationcited by 6
  6. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 6
  7. 2024Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimizationcited by 4
  8. 2025Online Robust Reinforcement Learning with General Function Approximationcited by 2
  9. 2024Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbationscited by 1