# Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System

arXiv: https://arxiv.org/abs/2408.15633 · HTML: https://mlsanity.com/paper/2408.15633/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 5 of its 28 references that are in the map.
- First posted on arXiv 28 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 5 of 5)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control](https://mlsanity.com/paper/1706.06491/) (2017)
- [Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control](https://mlsanity.com/paper/1910.12047/) (2019)
- [Python-Based Reinforcement Learning on Simulink Models](https://mlsanity.com/paper/2405.08567/) (2024)
- [Blue River Controls: A toolkit for Reinforcement Learning Control Systems on Hardware](https://mlsanity.com/paper/2001.02254/) (2020)

## Built on by (most cited first, 6 of 6)

- [The Crucial Role of Problem Formulation in Real-World Reinforcement Learning](https://mlsanity.com/paper/2503.20442/) (2025)
- [Multi-Objective Reinforcement Learning for Energy-Efficient Industrial Control](https://mlsanity.com/paper/2505.07607/) (2025)
- [Chebyshev Policies and the Mountain Car Problem: Reinforcement Learning for Low-Dimensional Control Tasks](https://mlsanity.com/paper/2605.22305/) (2026)
- [Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System](https://mlsanity.com/paper/2607.03125/) (2026)
- [Sample-Efficient Pareto Front Modeling for Energy-Aware Reinforcement Learning Using Bayesian Optimization](https://mlsanity.com/paper/2607.03140/) (2026)
- [Safe Reinforcement Learning using Ideas from Model Predictive Control](https://mlsanity.com/paper/2607.07252/) (2026)

## Abstract (arXiv)

This study conducts a comparative analysis of Model Predictive Control (MPC) and Proximal Policy Optimization (PPO), a Deep Reinforcement Learning (DRL) algorithm, applied to a 1-Degree of Freedom (DOF) Quanser Aero 2 system. Classical control techniques such as MPC and Linear Quadratic Regulator (LQR) are widely used due to their theoretical foundation and practical effectiveness. However, with advancements in computational techniques and machine learning, DRL approaches like PPO have gained traction in solving optimal control problems through environment interaction. This paper systematically evaluates the dynamic response characteristics of PPO and MPC, comparing their performance, computational resource consumption, and implementation complexity. Experimental results show that while LQR achieves the best steady-state accuracy, PPO excels in rise-time and adaptability, making it a promising approach for applications requiring rapid response and adaptability. Additionally, we have established a baseline for future RL-related research on this specific testbed. We also discuss the strengths and limitations of each control strategy, providing recommendations for selecting appropriate controllers for real-world scenarios.