mlsanityData to 14 Oct 2026

A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms

arXiv 2406.14753 ↗First posted 20 Jun 2024cs.LG stat.MEcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We devise a control-theoretic reinforcement learning approach to support direct learning of the optimal policy. We establish various theoretical properties of our approach, such as convergence and optimality of our analog of the Bellman operator and Q-learning, a new control-policy-variable gradient theorem, and a specific gradient ascent algorithm based on this theorem within the context of a specific control-theoretic framework. We empirically evaluate the performance of our control theoretic approach on several classical reinforcement learning tasks, demonstrating significant improvements in solution quality, sample complexity, and running time of our approach over state-of-the-art methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 43 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

17 of 43 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  7. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  8. 2023Finetuning Offline World Models in the Real Worldcited by 41
  9. 2023Adaptive Policy Learning for Offline-to-Online Reinforcement Learningcited by 40
  10. 2023Action-Quantized Offline Reinforcement Learning for Robotic Skill Learningcited by 36
  11. 2023HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulationcited by 29
  12. 2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learningcited by 14
5 more
  1. 2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 13
  2. 2023Optimal Attack and Defense for Reinforcement Learningcited by 12
  3. 2023Probabilistic Constraint for Safety-Critical Reinforcement Learningcited by 11
  4. 2023Model-based Offline Reinforcement Learning with Local Misspecificationcited by 5
  5. 2020Automatic differentiation of Sylvester, Lyapunov, and algebraic Riccati equationscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Overcoming the Curse of Dimensionality in Reinforcement Learning Through Approximate Factorizationcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2023Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization
  2. 2025Control and Reinforcement Learning through the Lens of Optimization: An Algorithmic Perspective
  3. 2026Mathematical methods of reinforcement learning
  4. 2024A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challenges
  5. 2022On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency
  6. 2026Foundations of Reinforcement Learning and Control:Connections and New Perspectives
  7. 2024The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementations
  8. 2026Deep Reinforcement Learning: From First Principles to Reasoning Models ↗