# Python-Based Reinforcement Learning on Simulink Models

arXiv: https://arxiv.org/abs/2405.08567 · HTML: https://mlsanity.com/paper/2405.08567/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 3 of its 12 references that are in the map.
- First posted on arXiv 14 May 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 3 of 3)

- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments](https://mlsanity.com/paper/2005.10619/) (2020)
- [Blue River Controls: A toolkit for Reinforcement Learning Control Systems on Hardware](https://mlsanity.com/paper/2001.02254/) (2020)

## Built on by (most cited first, 9 of 9)

- [Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System](https://mlsanity.com/paper/2408.15633/) (2024)
- [The Crucial Role of Problem Formulation in Real-World Reinforcement Learning](https://mlsanity.com/paper/2503.20442/) (2025)
- [Multi-Objective Reinforcement Learning for Energy-Efficient Industrial Control](https://mlsanity.com/paper/2505.07607/) (2025)
- [Deep Reinforcement Learning for Optimizing Inverter Control: Fixed and Adaptive Gain Tuning Strategies for Power System Stability](https://mlsanity.com/paper/2411.01451/) (2024)
- [Chebyshev Policies and the Mountain Car Problem: Reinforcement Learning for Low-Dimensional Control Tasks](https://mlsanity.com/paper/2605.22305/) (2026)
- [Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System](https://mlsanity.com/paper/2607.03125/) (2026)
- [Anticipatory Reinforcement Learning for Trajectory Tracking](https://mlsanity.com/paper/2607.03132/) (2026)
- [Sample-Efficient Pareto Front Modeling for Energy-Aware Reinforcement Learning Using Bayesian Optimization](https://mlsanity.com/paper/2607.03140/) (2026)
- [Safe Reinforcement Learning using Ideas from Model Predictive Control](https://mlsanity.com/paper/2607.07252/) (2026)

## Abstract (arXiv)

This paper proposes a framework for training Reinforcement Learning agents using Python in conjunction with Simulink models. Leveraging Python's superior customization options and popular libraries like Stable Baselines3, we aim to bridge the gap between the established Simulink environment and the flexibility of Python for training bleeding edge agents. Our approach is demonstrated on the Quanser Aero 2, a versatile dual-rotor helicopter. We show that policies trained on Simulink models can be seamlessly transferred to the real system, enabling efficient development and deployment of Reinforcement Learning agents for control tasks. Through systematic integration steps, including C-code generation from Simulink, DLL compilation, and Python interface development, we establish a robust framework for training agents on Simulink models. Experimental results demonstrate the effectiveness of our approach, surpassing previous efforts and highlighting the potential of combining Simulink with Python for Reinforcement Learning research and applications.