mlsanityData to 14 Oct 2026

Learning Robust Policies via Interpretable Hamilton-Jacobi Reachability-Guided Disturbances

arXiv 2409.19746 ↗First posted 29 Sep 2024cs.ROcited by 1

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Deep Reinforcement Learning (RL) has shown remarkable success in robotics with complex and heterogeneous dynamics. However, its vulnerability to unknown disturbances and adversarial attacks remains a significant challenge. In this paper, we propose a robust policy training framework that integrates model-based control principles with adversarial RL training to improve robustness without the need for external black-box adversaries. Our approach introduces a novel Hamilton-Jacobi reachability-guided disturbance for adversarial RL training, where we use interpretable worst-case or near-worst-case disturbances as adversaries against the robust policy. We evaluated its effectiveness across three distinct tasks: a reach-avoid game in both simulation and real-world settings, and a highly dynamic quadrotor stabilization task in simulation. We validate that our learned critic network is consistent with the ground-truth HJ value function, while the policy network shows comparable performance with other learning-based methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 49 references that are in the map. First posted on arXiv 29 Sep 2024.

Builds on

26 of 49 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Robust Adversarial Reinforcement Learningcited by 451
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2023Extreme Parkour with Legged Robotscited by 164
  5. 2020DeepReach: A Deep Learning Approach to High-Dimensional Reachabilitycited by 91
  6. 2021Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Controlcited by 77
  7. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 75
  8. 2020Robust Reinforcement Learning using Adversarial Populationscited by 66
  9. 2024WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contactscited by 61
  10. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 57
  11. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 50
  12. 2019Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotorscited by 45
14 more
  1. 2014Reach-Avoid Problems with Time-Varying Dynamics, Targets and Constraintscited by 44
  2. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 36
  3. 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 28
  4. 2022ISAACS: Iterative Soft Adversarial Actor-Critic for Safetycited by 28
  5. 2023What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discoverycited by 27
  6. 2024Hamilton-Jacobi Reachability in Reinforcement Learning: A Surveycited by 16
  7. 2022OptimizedDP: An Efficient, User-friendly Library For Optimal Control and Dynamic Programmingcited by 11
  8. 2023Robust Adversarial Reinforcement Learning via Bounded Rationality Curriculacited by 10
  9. 2023Guardians as You Fall: Active Mode Transition for Safe Fallingcited by 6
  10. 2016Exact and Efficient Hamilton-Jacobi-based Guaranteed Safety Analysis via System Decompositioncited by 3
  11. 2023Multi-Agent Reach-Avoid Games: Two Attackers Versus One Defender and Mixed Integer Programmingcited by 2
  12. 2024Providing Safety Assurances for Systems with Unknown Dynamicscited by 2
  13. 2024Robust Control Lyapunov-Value Functions for Nonlinear Disturbed Systemscited by 1
  14. 2024Robust Control using Control Lyapunov Function and Hamilton-Jacobi Reachabilitycited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust Adversarial Policy Optimization Under Dynamics Uncertainty
  2. 2026Post-Hoc Robustness for Model-Based Reinforcement Learning
  3. 2024Training Verifiably Robust Agents Using Set-Based Reinforcement Learning
  4. 2026LEAP-CBF: A Safety Filter for Uncertain Systems with Least-Effort Adversarial Potentials
  5. 2026Taming the Adversary: A Cost-to-Disturbance Ratio Approach to Adversarial Reinforcement Learning
  6. 2026Robust Adversarial Reinforcement Learning with Risk Sensitivity and Critic Consistency Regularization
  7. 2026RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning
  8. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models