mlsanityData to 14 Oct 2026

A novel agent with formal goal-reaching guarantees: an experimental study with a mobile robot

arXiv 2409.14867 ↗First posted 23 Sep 2024cs.RO cs.AI math.DScited by 1

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Reinforcement Learning (RL) has been shown to be effective and convenient for a number of tasks in robotics. However, it requires the exploration of a sufficiently large number of state-action pairs, many of which may be unsafe or unimportant. For instance, online model-free learning can be hazardous and inefficient in the absence of guarantees that a certain set of desired states will be reached during an episode. An increasingly common approach to address safety involves the addition of a shielding system that constrains the RL actions to a safe set of actions. In turn, a difficulty for such frameworks is how to effectively couple RL with the shielding system to make sure the exploration is not excessively restricted. This work presents a novel safe model-free RL agent called Critic As Lyapunov Function (CALF) and showcases how CALF can be used to improve upon control baselines in robotics in an efficient and convenient fashion while ensuring guarantees of stable goal reaching. The latter is a crucial part of safety, as seen generally. With CALF all state-action pairs remain explorable and yet reaching of desired goal states is formally guaranteed. Formal analysis is provided that shows the goal stabilization-ensuring properties of CALF and a set of real-world and numerical experiments with a non-holonomic wheeled mobile robot (WMR) TurtleBot3 Burger confirmed the superiority of CALF over such a well-established RL agent as proximal policy optimization (PPO), and a modified version of SARSA in a few-episode setting in terms of attained total cost.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 30 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

21 of 30 references.

References · 21

  1. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  2. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  3. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  4. 2020Projection-Based Constrained Policy Optimizationcited by 203
  5. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  6. 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 113
  7. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  8. 2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 59
  9. 2020Actor-Critic Reinforcement Learning for Control with Stability Guaranteecited by 45
  10. 2019Learning Human Objectives by Evaluating Hypothetical Behaviorcited by 41
  11. 2020Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimizationcited by 35
  12. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 35
9 more
  1. 2019Practical Reinforcement Learning of Stabilizing Economic MPCcited by 17
  2. 2020Infinite-Horizon Differentiable Model Predictive Controlcited by 17
  3. 2020Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robotcited by 17
  4. 2019Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learningcited by 13
  5. 2020Safe reinforcement learning for probabilistic reachability and safety specifications: A Lyapunov-based approachcited by 13
  6. 2022Model-free Neural Lyapunov Control for Safe Robot Navigationcited by 6
  7. 2024Critic as Lyapunov function (CALF): a model-free, stability-ensuring agentcited by 4
  8. 2020Deductive Stability Proofs for Ordinary Differential Equationscited by 3
  9. 2023A Multiplicative Value Function for Safe and Efficient Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Vision-based Goal-Reaching Control for Mobile Robots Using a Hierarchical Learning Frameworkcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks
  2. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  3. 2026Learning When to Act: Communication-Efficient Reinforcement Learning via Run-Time Assurance
  4. 2026Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning
  5. 2026Sampling-Based Safe Reinforcement Learning
  6. 2024Realizable Continuous-Space Shields for Safe Reinforcement Learning
  7. 2026Learning over Forward-Invariant Policy Classes: Reinforcement Learning without Safety Concerns
  8. 2026Shielded Reinforcement Learning Under Dynamic Temporal Logic Constraints