# Stable and Safe Human-aligned Reinforcement Learning through Neural Ordinary Differential Equations

arXiv: https://arxiv.org/abs/2401.13148 · HTML: https://mlsanity.com/paper/2401.13148/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 23 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 20 of 20)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions](https://mlsanity.com/paper/2109.06697/) (2021)
- [Reachability Constrained Reinforcement Learning](https://mlsanity.com/paper/2205.07536/) (2022)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [Actor-Critic Reinforcement Learning for Control with Stability Guarantee](https://mlsanity.com/paper/2004.14288/) (2020)
- [Feasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safety](https://mlsanity.com/paper/2105.10682/) (2021)
- [Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithm](https://mlsanity.com/paper/2210.07573/) (2022)
- [Physics-Regulated Deep Reinforcement Learning: Invariant Embeddings](https://mlsanity.com/paper/2305.16614/) (2023)
- [Volumetric Semantically Consistent 3D Panoptic Mapping](https://mlsanity.com/paper/2309.14737/) (2023)
- [Learning Robust State Observers using Neural ODEs (longer version)](https://mlsanity.com/paper/2212.00866/) (2022)
- [QuaDUE-CCM: Interpretable Distributional Reinforcement Learning using Uncertain Contraction Metrics for Precise Quadrotor Trajectory Tracking](https://mlsanity.com/paper/2207.07789/) (2022)
- [Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control](https://mlsanity.com/paper/2302.11694/) (2023)
- [Robust Barrier Functions for a Fully Autonomous, Remotely Accessible Swarm-Robotics Testbed](https://mlsanity.com/paper/1909.02966/) (2019)
- [Safety Verification and Controller Synthesis for Systems with Input Constraints](https://mlsanity.com/paper/2204.09386/) (2022)
- [Physical Deep Reinforcement Learning Towards Safety Guarantee](https://mlsanity.com/paper/2303.16860/) (2023)
- [On-ramp and Off-ramp Traffic Flows Estimation Based on A Data-driven Transfer Learning Framework](https://mlsanity.com/paper/2308.03538/) (2023)
- [Model-Free Verification for Neural Network Controlled Systems](https://mlsanity.com/paper/2312.08293/) (2023)

## Built on by (most cited first, 3 of 3)

- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [Exploring CausalWorld: Enhancing robotic manipulation via knowledge transfer and curriculum learning](https://mlsanity.com/paper/2403.17266/) (2024)
- [Graph Neural ODE Digital Twins for Control-Oriented Reactor Thermal-Hydraulic Forecasting Under Partial Observability](https://mlsanity.com/paper/2604.07292/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) excels in applications such as video games, but ensuring safety as well as the ability to achieve the specified goals remains challenging when using RL for real-world problems, such as human-aligned tasks where human safety is paramount. This paper provides safety and stability definitions for such human-aligned tasks, and then proposes an algorithm that leverages neural ordinary differential equations (NODEs) to predict human and robot movements and integrates the control barrier function (CBF) and control Lyapunov function (CLF) with the actor-critic method to help to maintain the safety and stability for human-aligned tasks. Simulation results show that the algorithm helps the controlled robot to reach the desired goal state with fewer safety violations and better sample efficiency compared to other methods in a human-aligned task.