mlsanityData to 13 Oct 2026

A Tour of Reinforcement Learning: The View from Continuous Control

arXiv 1806.09460 ↗First posted 25 Jun 2018math.OC cs.LG stat.MLcited by 130

Not on the map. Why?

Cited by 35 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. The manuscript describes how merging techniques from learning theory and control can provide non-asymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. This survey concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and control might be combined to approach these challenges.

Key facts. Cited by 130 papers in the map (data through 13 Oct 2026). First posted on arXiv 25 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 27; 2022: 25; 2023: 19; 2024: 20; 2025: 21; 2026: 17.

130 papers.

Sort:

By area:

  1. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 338
  2. 2021Enhancing Computational Fluid Dynamics with Machine Learningcited by 137
  3. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  4. 2021Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Controlcited by 68
  5. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
  6. 2023The transformative potential of machine learning for experiments in fluid mechanicscited by 26
  7. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 26
  8. 2022Safety Certification for Stochastic Systems via Neural Barrier Functionscited by 22
  9. 2024Learning Quadruped Locomotion Using Differentiable Simulationcited by 22
  10. 2019On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulatorcited by 20
  11. 2021Linear Systems can be Hard to Learncited by 20
  12. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 19
38 more
  1. 2021Concave Utility Reinforcement Learning with Zero-Constraint Violationscited by 18
  2. 2021Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexitycited by 15
  3. 2021Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learningcited by 15
  4. 2021Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systemscited by 15
  5. 2021Patterns, predictions, and actions: A story about machine learningcited by 14
  6. 2021Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear Systemcited by 13
  7. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 13
  8. 2023Model-free tracking control of complex dynamical trajectories with machine learningcited by 13
  9. 2023Sample-Efficient Linear Representation Learning from Non-IID Non-Isotropic Datacited by 12
  10. 2022Regret Lower Bounds for Learning Linear Quadratic Gaussian Systemscited by 11
  11. 2022Turnpike in optimal control of PDEs, ResNets, and beyondcited by 10
  12. 2022Learning to Control Linear Systems can be Hardcited by 10
  13. 2023Learning the Kalman Filter with Fine-Grained Sample Complexitycited by 9
  14. 2022Thompson Sampling Achieves tilde O(√T) Regret in Linear Quadratic Controlcited by 8
  15. 2024A Pontryagin Perspective on Reinforcement Learningcited by 7
  16. 2022Prospective Learning: Principled Extrapolation to the Futurecited by 6
  17. 2022Comparative analysis of machine learning methods for active flow controlcited by 6
  18. 2022Kullback-Leibler control for discrete-time nonlinear systems on continuous spacescited by 6
  19. 2022Modern Machine Learning Tools for Monitoring and Control of Industrial Processes: A Surveycited by 6
  20. 2022Towards a Theoretical Foundation of Policy Optimization for Learning Control Policiescited by 6
  21. 2024Machine learning for industrial sensing and control: A survey and practical perspectivecited by 6
  22. 2024Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systemscited by 6
  23. 2021Chance-Constrained Active Inferencecited by 5
  24. 2021Successive Convex Approximation Based Off-Policy Optimization for Constrained Reinforcement Learningcited by 5
  25. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  26. 2022Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Controlcited by 5
  27. 2022KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systemscited by 5
  28. 2024Data Assimilation in Chaotic Systems Using Deep Reinforcement Learningcited by 5
  29. 2024Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidencecited by 5
  30. 2024Decision Transformer as a Foundation Model for Partially Observable Continuous Controlcited by 5
  31. 2024Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programmingcited by 5
  32. 2025The Sample Complexity of Online Reinforcement Learning: A Multi-model Perspectivecited by 5
  33. 2021Control-Tutored Reinforcement Learning: Towards the Integration of Data-Driven and Model-Based Controlcited by 4
  34. 2022When Bioprocess Engineering Meets Machine Learning: A Survey from the Perspective of Automated Bioprocess Developmentcited by 4
  35. 2022Neural ODEs as Feedback Policies for Nonlinear Optimal Controlcited by 4
  36. 2023Global Convergence of Receding-Horizon Policy Search in Learning Estimator Designscited by 4
  37. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  38. 2021DiffLoop: Tuning PID controllers by differentiating through the feedback loopcited by 3

50 most cited of 130 All 130 (JSON)