# Control Synthesis with Reinforcement Learning: A Modeling Perspective

arXiv: https://arxiv.org/abs/2510.25063 · HTML: https://mlsanity.com/paper/2510.25063/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 24 of 24)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems](https://mlsanity.com/paper/1808.00924/) (2018)
- [BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators](https://mlsanity.com/paper/1906.01728/) (2019)
- [Towards Generalization and Simplicity in Continuous Control](https://mlsanity.com/paper/1703.02660/) (2017)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learning](https://mlsanity.com/paper/1802.06480/) (2018)
- [Auto-Tuned Sim-to-Real Transfer](https://mlsanity.com/paper/2104.07662/) (2021)
- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [Symmetry in Neural Network Parameter Spaces](https://mlsanity.com/paper/2506.13018/) (2025)
- [Lyapunov Design for Robust and Efficient Robotic Reinforcement Learning](https://mlsanity.com/paper/2208.06721/) (2022)
- [Chance Constrained Policy Optimization for Process Control and Optimization](https://mlsanity.com/paper/2008.00030/) (2020)
- [Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots](https://mlsanity.com/paper/2303.09597/) (2023)
- [Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization](https://mlsanity.com/paper/2306.01276/) (2023)
- [Symmetry-enforcing neural networks with applications to constitutive modeling](https://mlsanity.com/paper/2312.13511/) (2023)
- [Integrating Model-based Control and RL for Sim2Real Transfer of Tight Insertion Policies](https://mlsanity.com/paper/2505.11858/) (2025)
- [Examining the simulation-to-reality gap of a wheel loader digging in deformable terrain](https://mlsanity.com/paper/2310.05765/) (2023)
- [AUGCAL: Improving Sim2Real Adaptation by Uncertainty Calibration on Augmented Synthetic Images](https://mlsanity.com/paper/2312.06106/) (2023)
- [Tight Certified Robustness via Min-Max Representations of ReLU Neural Networks](https://mlsanity.com/paper/2310.04916/) (2023)

## Built on by (most cited first, 1 of 1)

- [Zero-shot Transfer of Reinforcement Learning Control Policies for the Swing-Up and Stabilization of a Cart-Pole System](https://mlsanity.com/paper/2606.22145/) (2026)

## Abstract (arXiv)

Controllers designed with reinforcement learning can be sensitive to model mismatch. We demonstrate that designing such controllers in a virtual simulation environment with an inaccurate model is not suitable for deployment in a physical setup. Controllers designed using an accurate model is robust against disturbance and small mismatch between the physical setup and the mathematical model derived from first principles; while a poor model results in a controller that performs well in simulation but fails in physical experiments. Sensitivity analysis is used to justify these discrepancies and an empirical region of attraction estimation help us visualize their robustness.