mlsanityData to 14 Oct 2026

To Learn or Not to Learn: A Litmus Test for Using Reinforcement Learning in Control

arXiv 2604.11463 ↗First posted 13 Apr 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning (RL) can be a powerful alternative to classical control methods when standard model-based control is insufficient, e.g., when deriving a suitable model is intractable or impossible. In many cases, however, the choice between model-based and RL-based control is not obvious. Due to the high computational costs of training RL agents, RL-based control should be limited to cases where it is expected to yield superior results compared to model-based control. To the best of our knowledge, there exists no approach to quantify the benefit of RL-based control that does not require RL training. In this work, we present a computationally efficient, purely simulation-based litmus test predicting whether RL-based control is superior to model-based control. Our test evaluates the suitability of the given model for model-based control by analyzing the impact of model uncertainties on the control problem. For this, we use reachset-conformant model identification combined with simulation-based analysis. This is followed by a learnability evaluation of the uncertainties based on correlation analysis. This two-part analysis enables an informed decision on the suitability of RL for a control problem without training an RL agent. We apply our test to several benchmarks, demonstrating its applicability to a wide range of control problems and highlight the potential to save computational resources.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 31 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

17 of 31 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  7. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  8. 2018An Introduction to Deep Reinforcement Learningcited by 206
  9. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  10. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 50
  11. 2013The Randomized Dependence Coefficientcited by 26
  12. 2019Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Controlcited by 22
5 more
  1. 2023Reinforcement Learning Versus Model Predictive Control on Greenhouse Climate Controlcited by 12
  2. 2022Learning to Control Linear Systems can be Hardcited by 11
  3. 2024PC-Gym: Benchmark Environments For Process Control Problemscited by 10
  4. 2021Model Predictive Control of a Vehicle using Koopman Operatorcited by 6
  5. 2024Reachset-Conformant System Identificationcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022A Survey on Model-based Reinforcement Learning
  2. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations
  3. 2026RLLBC-Lib: An Educational Code Library for Reinforcement Learning and Learning-Based Control
  4. 2026A Systematic Review and Taxonomy of Reinforcement Learning-Model Predictive Control Integration for Linear Systems
  5. 2025Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
  6. 2024Reinforcement Learning for Motor Control: A Comprehensive Review
  7. 2025Statistical and Algorithmic Foundations of Reinforcement Learning
  8. 2026RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training