# Flow-based Domain Randomization for Learning and Sequencing Robotic Skills

arXiv: https://arxiv.org/abs/2502.01800 · HTML: https://mlsanity.com/paper/2502.01800/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 3 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 30 of 37)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare](https://mlsanity.com/paper/2212.06870/) (2022)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [Assessing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1810.12282/) (2018)
- [The Ingredients of Real-World Robotic Reinforcement Learning](https://mlsanity.com/paper/2004.12570/) (2020)
- [Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models](https://mlsanity.com/paper/2401.03360/) (2023)
- [BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators](https://mlsanity.com/paper/1906.01728/) (2019)
- [Understanding Domain Randomization for Sim-to-real Transfer](https://mlsanity.com/paper/2110.03239/) (2021)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [RL for Latent MDPs: Regret Guarantees and a Lower Bound](https://mlsanity.com/paper/2102.04939/) (2021)
- [Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards](https://mlsanity.com/paper/1906.05841/) (2019)
- [IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality](https://mlsanity.com/paper/2305.17110/) (2023)
- [Policy Transfer with Strategy Optimization](https://mlsanity.com/paper/1810.05751/) (2018)
- [Assessing Transferability from Simulation to Reality for Reinforcement Learning](https://mlsanity.com/paper/1907.04685/) (2019)
- [A Modular Robotic Arm Control Stack for Research: Franka-Interface and FrankaPy](https://mlsanity.com/paper/2011.02398/) (2020)
- [FORGE: Force-Guided Exploration for Robust Contact-Rich Manipulation under Uncertainty](https://mlsanity.com/paper/2408.04587/) (2024)
- [A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning](https://mlsanity.com/paper/2102.13176/) (2021)
- [Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamics](https://mlsanity.com/paper/2008.06686/) (2020)
- [Domain Randomization via Entropy Maximization](https://mlsanity.com/paper/2311.01885/) (2023)
- [Partially Observable Task and Motion Planning with Uncertainty and Risk Awareness](https://mlsanity.com/paper/2403.10454/) (2024)
- [Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks](https://mlsanity.com/paper/2206.06282/) (2022)

## Built on by (most cited first, 4 of 4)

- [Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness](https://mlsanity.com/paper/2603.21017/) (2026)
- [Online Adaptive Reinforcement Learning with Echo State Networks for Non-Stationary Dynamics](https://mlsanity.com/paper/2602.06326/) (2026)
- [DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning](https://mlsanity.com/paper/2607.16090/) (2026)
- [Generalizing HVAC Control With Domain Randomized Reinforcement Learning](https://mlsanity.com/paper/2609.05822/) (2026)

## Abstract (arXiv)

Domain randomization in reinforcement learning is an established technique for increasing the robustness of control policies trained in simulation. By randomizing environment properties during training, the learned policy can become robust to uncertainties along the randomized dimensions. While the environment distribution is typically specified by hand, in this paper we investigate automatically discovering a sampling distribution via entropy-regularized reward maximization of a normalizing-flow-based neural sampling distribution. We show that this architecture is more flexible and provides greater robustness than existing approaches that learn simpler, parameterized sampling distributions, as demonstrated in six simulated and one real-world robotics domain. Lastly, we explore how these learned sampling distributions, combined with a privileged value function, can be used for out-of-distribution detection in an uncertainty-aware multi-step manipulation planner.