mlsanityData to 14 Oct 2026

Coordinating Planning and Tracking in Layered Control Policies via Actor-Critic Learning

arXiv 2408.01639 ↗First posted 3 Aug 2024eess.SY cs.LG cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We propose a reinforcement learning (RL)-based algorithm to jointly train (1) a trajectory planner and (2) a tracking controller in a layered control architecture. Our algorithm arises naturally from a rewrite of the underlying optimal control problem that lends itself to an actor-critic learning approach. By explicitly learning a textitdual network to coordinate the interaction between the planning and tracking layers, we demonstrate the ability to achieve an effective consensus between the two components, leading to an interpretable policy. We theoretically prove that our algorithm converges to the optimal dual network in the Linear Quadratic Regulator (LQR) setting and empirically validate its applicability to nonlinear systems through simulation experiments on a unicycle model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 32 references that are in the map. First posted on arXiv 3 Aug 2024.

Builds on

17 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Actor-Critic Physics-informed Neural Lyapunov Control13 Mar

Other references · 16

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  4. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  5. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  6. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  7. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  8. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
  9. 2017Least-Squares Temporal Difference Learning for the Linear Quadratic Regulatorcited by 30
  10. 2022CACTO: Continuous Actor-Critic with Trajectory Optimization -- Towards global optimalitycited by 12
  11. 2020Unified Multi-Rate Control: from Low Level Actuation to High Level Planningcited by 8
  12. 2024Towards a Theory of Control Architecture: A quantitative framework for layered multi-rate controlcited by 4
4 more
  1. 2022Multi-Rate Planning and Control of Uncertain Nonlinear Systems: Model Predictive Control and Control Lyapunov Functionscited by 3
  2. 2023Augmented Lagrangian Methods as Layered Control Architecturescited by 3
  3. 2023A Data-Driven Approach to Synthesizing Dynamics-Aware Trajectories for Underactuated Robotic Systemscited by 2
  4. 2024Why Change Your Controller When You Can Change Your Planner: Drag-Aware Trajectory Generation for Quadrotor Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Foundations of Reinforcement Learning and Control:Connections and New Perspectives
  2. 2024AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control
  3. 2024Natural Policy Gradient and Actor Critic Methods for Constrained Multi-Task Reinforcement Learning
  4. 2024Compatible Gradient Approximations for Actor-Critic Algorithms
  5. 2025Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returns
  6. 2025Distributions as Actions: A Unified Framework for Diverse Action Spaces
  7. 2025Reinforcement Networks: novel framework for collaborative Multi-Agent Reinforcement Learning tasks
  8. 2025A Communication-Efficient Decentralized Actor-Critic Algorithm