mlsanityData to 14 Oct 2026

A Novel Multi-Timescale Stability-Preserving Hierarchical Reinforcement Learning Controller Framework for Adaptive Control in High-Dimensional Dynamical Systems

arXiv 2510.22420 ↗First posted 25 Oct 2025cs.RO cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Controlling high-dimensional stochastic systems, critical in robotics, autonomous vehicles, and hyperchaotic systems, faces the curse of dimensionality, lacks temporal abstraction, and often fails to ensure stochastic stability. To overcome these limitations, this study introduces the Multi-Timescale Lyapunov-Constrained Hierarchical Reinforcement Learning (MTLHRL) framework. MTLHRL integrates a hierarchical policy within a semi-Markov Decision Process (SMDP), featuring a high-level policy for strategic planning and a low-level policy for reactive control, which effectively manages complex, multi-timescale decision-making and reduces dimensionality overhead. Stability is rigorously enforced using a neural Lyapunov function optimized via Lagrangian relaxation and multi-timescale actor-critic updates, ensuring mean-square boundedness or asymptotic stability in the face of stochastic dynamics. The framework promotes efficient and reliable learning through trust-region constraints and decoupled optimization. Extensive simulations on an 8D hyperchaotic system and a 5-DOF robotic manipulator demonstrate MTLHRL's empirical superiority. It significantly outperforms baseline methods in both stability and performance, recording the lowest error indices (e.g., Integral Absolute Error (IAE): 3.912 in hyperchaotic control and IAE: 1.623 in robotics), achieving faster convergence, and exhibiting superior disturbance rejection. MTLHRL offers a theoretically grounded and practically viable solution for robust control of complex stochastic systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 59 references that are in the map. First posted on arXiv 25 Oct 2025.

Builds on

17 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning15 May

Other references · 16

  1. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  2. 2020PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learningcited by 92
  3. 2022Reachability Constrained Reinforcement Learningcited by 59
  4. 2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 52
  5. 2020Actor-Critic Reinforcement Learning for Control with Stability Guaranteecited by 45
  6. 2021Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equationscited by 30
  7. 2025A Systematic Evaluation of LLM Strategies for Mental Health Text Analysis: Fine-tuning vs. Prompt Engineering vs. RAGcited by 15
  8. 2023Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approachcited by 13
  9. 2024Continuous-Time State Estimation Methods in Robotics: A Surveycited by 10
  10. 2016Ultra High-Dimensional Nonlinear Feature Selection for Big Biological Datacited by 7
  11. 2025Time Series Embedding Methods for Classification Tasks: A Reviewcited by 7
  12. 2024Risk-Aware Robotics: Tail Risk Measures in Planning, Control, and Verificationcited by 4
4 more
  1. 2024Optimal Inferential Control of Convolutional Neural Networkscited by 3
  2. 2025Optimizing Flamelet Generated Manifold Models: A Machine Learning Performance Studycited by 2
  3. 2025Interpretable Machine Learning for Predicting Startup Funding, Patenting, and Exitscited by 2
  4. 2023Exploring reinforcement learning techniques for discrete and continuous control tasks in the MuJoCo environmentcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Stochastic Reinforcement Learning with Stability Guarantees for Control of Unknown Nonlinear Systems
  2. 2024Time-Constrained Robust MDPs
  3. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
  4. 2023Learning to Stabilize High-dimensional Unknown Systems Using Lyapunov-guided Exploration
  5. 2026Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control
  6. 2025MSACL: Multi-Step Actor-Critic Learning with Lyapunov Certificates for Exponentially Stabilizing Control
  7. 2025Neural Lyapunov Function Approximation with Self-Supervised Reinforcement Learning
  8. 2025Never Too Rigid to Reach: Adaptive Virtual Model Control with LLM- and Lyapunov-Based Reinforcement Learning ↗