BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators
Not on the map. Why?
Abstract (arXiv)
We introduce BayesSim, a framework for robotics simulations allowing a full Bayesian treatment for the parameters of the simulator. As simulators become more sophisticated and able to represent the dynamics more accurately, fundamental problems in robotics such as motion planning and perception can be solved in simulation and solutions transferred to the physical robot. However, even the most complex simulator might still not be able to represent reality in all its details either due to inaccurate parametrization or simplistic assumptions in the dynamic models. BayesSim provides a principled framework to reason about the uncertainty of simulation parameters. Given a black box simulator (or generative model) that outputs trajectories of state and action pairs from unknown simulation parameters, followed by trajectories obtained with a physical robot, we develop a likelihood-free inference method that computes the posterior distribution of simulation parameters. This posterior can then be used in problems where Sim2Real is critical, for example in policy search. We compare the performance of BayesSim in obtaining accurate posteriors in a number of classical control and robotics problems. Results show that the posterior computed from BayesSim can be used for domain randomization outperforming alternative methods that randomize based on uniform priors.
Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jun 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2020: 1; 2021: 15; 2022: 17; 2023: 11; 2024: 9; 2025: 14; 2026: 11.
78 papers.
Sort:
By area:
- 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
- 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
- 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
- 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
- 2022Neural-Fly Enables Rapid Learning for Agile Flight in Strong Windscited by 80
- 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 78
- 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
- 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
- 2024GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMscited by 48
- 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
- 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
- 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
38 more
- 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 34
- 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
- 2022DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objectscited by 29
- 2020Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learningcited by 27
- 2024ASID: Active Exploration for System Identification in Robotic Manipulationcited by 27
- 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
- 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 24
- 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
- 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
- 2021Value Iteration in Continuous Actions, States and Timecited by 17
- 2021Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulationcited by 17
- 2023AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfercited by 17
- 2023Adversarial robustness of amortized Bayesian inferencecited by 17
- 2022DROPO: Sim-to-Real Transfer with Offline Domain Randomizationcited by 16
- 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
- 2023MoVie: Visual Model-Based Policy Adaptation for View Generalizationcited by 15
- 2023H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gapscited by 15
- 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 15
- 2021A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulationcited by 11
- 2024The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learningcited by 11
- 2021Dual Online Stein Variational Inference for Control and Dynamicscited by 10
- 2022Policy Adaptation from Foundation Model Feedbackcited by 10
- 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
- 2024Robot Learning with Super-Linear Scalingcited by 10
- 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learningcited by 10
- 2021Robust Value Iteration for Continuous Control Taskscited by 8
- 2021OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulationcited by 7
- 2022Weakly Supervised Correspondence Learningcited by 7
- 2022Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomizationcited by 7
- 2026ContactGaussian-WM: Learning Physics-Grounded World Model from Videoscited by 7
- 2021Sim and Real: Better Togethercited by 6
- 2021Continuous-Time Fitted Value Iteration for Robust Policiescited by 6
- 2022DiSECt: A Differentiable Simulator for Parameter Inference and Control in Robotic Cuttingcited by 6
- 2022Real2Sim2Real Transfer for Control of Cable-driven Robots via a Differentiable Physics Enginecited by 5
- 2022Online vs. Offline Adaptive Domain Randomization Benchmarkcited by 4
- 2023Fine-Tuning Generative Models as an Inference Method for Robotic Taskscited by 4
- 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skillscited by 4
- 2021BayesSimIG: Scalable Parameter Inference for Adaptive Domain Randomization with IsaacGymcited by 3
50 most cited of 78 All 78 (JSON)
- 2026EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning28 Sep 2026
- 2026VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RL25 Sep 2026
- 2026Adapting Generalist Vehicle Models for High-Speed MPC Across Terrains14 Jul 2026
- 2026Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement17 Jun 2026
- 2026TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation4 Jun 2026
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey29 Apr 2026
- 2026SPIRIT: Perceptive Shared Autonomy for Robust Robotic Manipulation under Deep Learning Uncertainty5 Mar 2026
- 2026D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping1 Mar 2026
- 2026Cooperative-Competitive Team Play of Real-World Craft Robots24 Feb 2026
- 2026Vid2Sid: Videos Can Help Close the Sim2Real Gap22 Feb 2026
- 2026ContactGaussian-WM: Learning Physics-Grounded World Model from Videos11 Feb 2026
- 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learning15 Dec 2025
38 more
- 2025Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation30 Oct 2025
- 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practices23 Oct 2025
- 2025ASBI: Leveraging Informative Real-World Data for Active Black-Box Simulator Tuning17 Oct 2025
- 2025Statistical Guarantees for Offline Domain Randomization11 Jun 2025
- 2025Model-based controller assisted domain randomization for transient vibration suppression of nonlinear powertrain system with parametric uncertainty28 Apr 2025
- 2025Neural Fidelity Calibration for Informative Sim-to-Real Adaptation11 Apr 2025
- 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation31 Mar 2025
- 2025An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer13 Mar 2025
- 2025A Multi-Fidelity Control Variate Approach for Policy Gradient Estimation7 Mar 2025
- 2025A Distributional Treatment of Real2Sim2Real for Object-Centric Agent Adaptation in Vision-Driven Deformable Linear Object Manipulation25 Feb 2025
- 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuning4 Feb 2025
- 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skills3 Feb 2025
- 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills3 Feb 2025
- 2024Robot Learning with Super-Linear Scaling2 Dec 2024
- 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learning28 Oct 2024
- 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications27 Oct 2024
- 2024GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs4 Oct 2024
- 2024DrEureka: Language Model Guided Sim-To-Real Transfer4 Jun 2024
- 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatch24 May 2024
- 2024The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learning6 May 2024
- 2024ASID: Active Exploration for System Identification in Robotic Manipulation18 Apr 2024
- 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents7 Feb 2024
- 2023Fine-Tuning Generative Models as an Inference Method for Robotic Tasks19 Oct 2023
- 2023Estimating Material Properties of Interacting Objects Using Sum-GP-UCB18 Oct 2023
- 2023BayRnTune: Adaptive Bayesian Domain Randomization via Strategic Fine-tuning16 Oct 2023
- 2023GenSim: Generating Robotic Simulation Tasks via Large Language Models2 Oct 2023
- 2023H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps22 Sep 2023
- 2023MoVie: Visual Model-Based Policy Adaptation for View Generalization3 Jul 2023
- 2023Cross-Domain Policy Adaptation via Value-Guided Data Filtering28 May 2023
- 2023Adversarial robustness of amortized Bayesian inference24 May 2023
- 2023Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robots7 Mar 2023
- 2023AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfer9 Feb 2023
- 2023Zero-shot Sim2Real Adaptation Across Environments8 Feb 2023
- 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning17 Dec 2022
- 2022Policy Adaptation from Foundation Model Feedback14 Dec 2022
- 2022Visual Reinforcement Learning with Self-Supervised 3D Representations13 Oct 2022
- 2022Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptation24 Sep 2022
- 2022Quantification before Selection: Active Dynamics Preference for Robust Reinforcement Learning23 Sep 2022
50 newest of 78 All 78 (JSON)
- 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
- 2024GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMscited by 48
- 2026ContactGaussian-WM: Learning Physics-Grounded World Model from Videoscited by 7
- 2026EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planningcited by 0
- 2026EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning28 Sep 2026
- 2026ContactGaussian-WM: Learning Physics-Grounded World Model from Videos11 Feb 2026
- 2024GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs4 Oct 2024
- 2023GenSim: Generating Robotic Simulation Tasks via Large Language Models2 Oct 2023
- 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learningcited by 10
- 2026Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancementcited by 1
- 2026VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RLcited by 0
- 2026VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RL25 Sep 2026
- 2026Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement17 Jun 2026
- 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learning15 Dec 2025
- 2022Policy Adaptation from Foundation Model Feedbackcited by 10
- 2022Policy Adaptation from Foundation Model Feedback14 Dec 2022
- 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
- 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
- 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
- 2022Neural-Fly Enables Rapid Learning for Agile Flight in Strong Windscited by 80
- 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 78
- 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
- 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
- 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
- 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
- 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
- 2021Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFingercited by 34
- 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
12 most cited of 67 All 67 (JSON)
- 2026Adapting Generalist Vehicle Models for High-Speed MPC Across Terrains14 Jul 2026
- 2026TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation4 Jun 2026
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey29 Apr 2026
- 2026SPIRIT: Perceptive Shared Autonomy for Robust Robotic Manipulation under Deep Learning Uncertainty5 Mar 2026
- 2026D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping1 Mar 2026
- 2026Cooperative-Competitive Team Play of Real-World Craft Robots24 Feb 2026
- 2026Vid2Sid: Videos Can Help Close the Sim2Real Gap22 Feb 2026
- 2025Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation30 Oct 2025
- 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practices23 Oct 2025
- 2025ASBI: Leveraging Informative Real-World Data for Active Black-Box Simulator Tuning17 Oct 2025
- 2025Statistical Guarantees for Offline Domain Randomization11 Jun 2025
- 2025Model-based controller assisted domain randomization for transient vibration suppression of nonlinear powertrain system with parametric uncertainty28 Apr 2025
12 newest of 67 All 67 (JSON)