Action Robust Reinforcement Learning and Applications in Continuous Control
Not on the map. Why?
Abstract (arXiv)
A policy is said to be robust if it maximizes the reward while considering a bad, or even adversarial, model. In this work we formalize two new criteria of robustness to action uncertainty. Specifically, we consider two scenarios in which the agent attempts to perform an action a, and (i) with probability α, an alternative adversarial action bar a is taken, or (ii) an adversary adds a perturbation to the selected action in the case of continuous action space. We show that our criteria are related to common forms of uncertainty in robotics domains, such as the occurrence of abrupt forces, and suggest algorithms in the tabular case. Building on the suggested algorithms, we generalize our approach to deep reinforcement learning (DRL) and provide extensive experiments in the various MuJoCo domains. Our experiments show that not only does our approach produce robust policies, but it also improves the performance in the absence of perturbations. This generalization indicates that action-robustness can be thought of as implicit regularization in RL problems.
Key facts. Cited by 167 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jan 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2020: 1; 2021: 22; 2022: 33; 2023: 37; 2024: 22; 2025: 27; 2026: 25.
167 papers.
Sort:
By area:
- 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
- 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
- 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
- 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
- 2022Robust Reinforcement Learning using Offline Datacited by 95
- 2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
- 2022Constrained Policy Optimization via Bayesian World Modelscited by 55
- 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 54
- 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
- 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
- 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
- 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 42
38 more
- 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
- 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
- 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 35
- 2023Robust Multi-Agent Reinforcement Learning with State Uncertaintycited by 35
- 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
- 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
- 2024Reward-Robust RLHF in LLMscited by 32
- 2021Robust Predictable Controlcited by 29
- 2023Policy Gradient for Rectangular Robust Markov Decision Processescited by 29
- 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 26
- 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
- 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 26
- 2023Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlationcited by 25
- 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
- 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
- 2025RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbationscited by 23
- 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
- 2023Robust Average-Reward Markov Decision Processescited by 22
- 2023Model-Free Robust Average-Reward Reinforcement Learningcited by 20
- 2021Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimizationcited by 18
- 2024Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learningcited by 17
- 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
- 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learningcited by 16
- 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
- 2025Horizon Generalization in Reinforcement Learningcited by 15
- 2021Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learningcited by 14
- 2022Are AlphaZero-like Agents Robust to Adversarial Perturbations?cited by 14
- 2021Multi-Task Federated Reinforcement Learning with Adversariescited by 13
- 2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12
- 2023Optimal Attack and Defense for Reinforcement Learningcited by 12
- 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 12
- 2022Robust Reinforcement Learning via Genetic Curriculumcited by 10
- 2023Safe Reinforcement Learning with Dual Robustnesscited by 10
- 2023Solving Long-run Average Reward Robust MDPs via Stochastic Gamescited by 10
- 2022Robust Learning from Observation with Model Misspecificationcited by 9
- 2022Robust Reinforcement Learning with Distributional Risk-averse formulationcited by 9
- 2022Provable Sim-to-real Transfer in Continuous Domain with Partial Observationscited by 9
- 2023Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guaranteescited by 9
50 most cited of 167 All 167 (JSON)
- 2026Learning Perturbation Robust Policies for LLM Agents with Stable Optimization28 Sep 2026
- 2026ASGARD: Action-Space Guard for UAV Resilience via Reinforcement Learning17 Sep 2026
- 2026Solving Robust POMDPs with Omega-regular Objectives via Partially Observable Stochastic Games25 Aug 2026
- 2026Online Security Learning in Cooperative Multi-Agent Systems under Hidden Byzantine Attacks6 Aug 2026
- 2026Robust General Utility for Reinforcement Learning4 Aug 2026
- 2026Training Language Models to Cooperate with Inference-Time Controllers26 Jul 2026
- 2026Entropy Regularization Improves Policy Robustness in Continuous-Time Reinforcement Learning3 Jul 2026
- 2026Neetyabhas: A Framework for Uncertainty-Aware Public Policy Optimization in Rational Agent-Based Models3 Jun 2026
- 2026Post-Hoc Robustness for Model-Based Reinforcement Learning2 Jun 2026
- 2026Bridging the Sim-to-Real Gap in Reinforcement Learning-Based Industrial Dispatching through Execution Semantics27 May 2026
- 2026Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learning18 May 2026
- 2026Robust Multi-Agent Path Finding under Observation Attacks: A Principled Adversarial-Plus-Smoothing Training Recipe12 May 2026
38 more
- 2026When Actions Disappear: Adversarial Action Removal in Self-Play Reinforcement Learning4 May 2026
- 2026Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking29 Apr 2026
- 2026Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Models20 Apr 2026
- 2026Robust Adversarial Policy Optimization Under Dynamics Uncertainty13 Apr 2026
- 2026ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving16 Mar 2026
- 2026Taming the Adversary: A Cost-to-Disturbance Ratio Approach to Adversarial Reinforcement Learning12 Mar 2026
- 2026Cross-Domain Policy Optimization via Bellman Consistency and Hybrid Critics12 Mar 2026
- 2026Fuz-RL: A Fuzzy-Guided Robust Framework for Safe Reinforcement Learning under Uncertainty24 Feb 2026
- 2026CDRL: A Reinforcement Learning Framework Inspired by Cerebellar Circuits and Dendritic Computational Strategies17 Feb 2026
- 2026TCRL: Temporal-Coupled Adversarial Training for Robust Constrained Reinforcement Learning in Worst-Case Scenarios13 Feb 2026
- 2026Formal Synthesis of Certifiably Robust Neural Lyapunov-Barrier Certificates5 Feb 2026
- 2026Beware Untrusted Simulators -- Reward-Free Backdoor Attacks in Reinforcement Learning4 Feb 2026
- 2026Bayesian Robust Financial Trading with Adversarial Synthetic Market Data14 Jan 2026
- 2025Tackling Snow-Induced Challenges: Safe Autonomous Lane-Keeping with Robust Reinforcement Learning15 Dec 2025
- 2025UACER: An Uncertainty-Adaptive Critic Ensemble Framework for Robust Adversarial Reinforcement Learning11 Dec 2025
- 2025Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defenses23 Oct 2025
- 2025Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Control15 Oct 2025
- 2025SAJA: A State-Action Joint Attack Framework on Multi-Agent Deep Reinforcement Learning15 Oct 2025
- 2025Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learning14 Oct 2025
- 2025Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning13 Oct 2025
- 2025Realistic CDSS Drug Dosing with End-to-end Recurrent Q-learning for Dual Vasopressor Control1 Oct 2025
- 2025Adversarial Diffusion for Robust Reinforcement Learning28 Sep 2025
- 2025RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations26 Sep 2025
- 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer23 Sep 2025
- 2025Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning18 Sep 2025
- 2025Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach22 Jul 2025
- 2025Action Robust Reinforcement Learning via Optimal Adversary Aware Policy Optimization4 Jul 2025
- 2025Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments26 Jun 2025
- 2025Meta Policy Switching for Secure UAV Deconfliction in Adversarial Airspace26 Jun 2025
- 2025Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluation20 Jun 2025
- 2025Viability of Future Actions: Robust Safety in Reinforcement Learning via Entropy Regularization12 Jun 2025
- 2025Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learning24 May 2025
- 2025Maximum Total Correlation Reinforcement Learning22 May 2025
- 2025TraCeS: Learning Per-Timestep Constraint-Violation Credit from Sparse Trajectory-Level Labels17 Apr 2025
- 2025Robust Deterministic Policy Gradient for Disturbance Attenuation and Its Application to Quadrotor Control28 Feb 2025
- 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning27 Feb 2025
- 2025Efficiently Solving Discounted MDPs with Predictions on Transition Matrices21 Feb 2025
- 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models18 Feb 2025
50 newest of 167 All 167 (JSON)
- 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
- 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
- 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
- 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
- 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
- 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
- 2024Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learningcited by 17
- 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learningcited by 16
- 2022Robust Learning from Observation with Model Misspecificationcited by 9
- 2023MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operatorcited by 9
- 2024Adversarially Robust Decision Transformercited by 4
- 2024Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbationscited by 1
12 most cited of 14 All 14 (JSON)
- 2025Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Control15 Oct 2025
- 2024Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbations25 Dec 2024
- 2024Adversarially Robust Decision Transformer25 Jul 2024
- 2024Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning14 Mar 2024
- 2023MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operator7 Dec 2023
- 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruption19 Oct 2023
- 2023Robust Offline Reinforcement Learning -- Certify the Confidence Interval28 Sep 2023
- 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage16 May 2023
- 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Belief13 Oct 2022
- 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning22 Sep 2022
- 2022Robust Imitation Learning against Variations in Environment Dynamics19 Jun 2022
- 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learning26 Apr 2022
12 newest of 14 All 14 (JSON)
- 2021Learning a Robust Multiagent Driving Policy for Traffic Congestion Reductioncited by 4
- 2026ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Drivingcited by 3
- 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 1
- 2024Autonomous vehicle decision and control through reinforcement learning with traffic flow randomizationcited by 0
- 2026ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving16 Mar 2026
- 2024Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization5 Mar 2024
- 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Survey12 Dec 2022
- 2021Learning a Robust Multiagent Driving Policy for Traffic Congestion Reduction3 Dec 2021
- 2024Reward-Robust RLHF in LLMscited by 32
- 2026Uncertainty-Aware Reward Discounting for Mitigating Reward Hackingcited by 1
- 2026Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking29 Apr 2026
- 2024Reward-Robust RLHF in LLMs18 Sep 2024
- 2025Adversarial Diffusion for Robust Reinforcement Learningcited by 4
- 2025Adversarial Diffusion for Robust Reinforcement Learning28 Sep 2025
- 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
- 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
- 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
- 2022Robust Reinforcement Learning using Offline Datacited by 95
- 2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
- 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 54
- 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
- 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
- 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 42
- 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
- 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
- 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 35
12 most cited of 141 All 141 (JSON)
- 2026ASGARD: Action-Space Guard for UAV Resilience via Reinforcement Learning17 Sep 2026
- 2026Solving Robust POMDPs with Omega-regular Objectives via Partially Observable Stochastic Games25 Aug 2026
- 2026Online Security Learning in Cooperative Multi-Agent Systems under Hidden Byzantine Attacks6 Aug 2026
- 2026Robust General Utility for Reinforcement Learning4 Aug 2026
- 2026Training Language Models to Cooperate with Inference-Time Controllers26 Jul 2026
- 2026Entropy Regularization Improves Policy Robustness in Continuous-Time Reinforcement Learning3 Jul 2026
- 2026Neetyabhas: A Framework for Uncertainty-Aware Public Policy Optimization in Rational Agent-Based Models3 Jun 2026
- 2026Post-Hoc Robustness for Model-Based Reinforcement Learning2 Jun 2026
- 2026Bridging the Sim-to-Real Gap in Reinforcement Learning-Based Industrial Dispatching through Execution Semantics27 May 2026
- 2026Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learning18 May 2026
- 2026Robust Multi-Agent Path Finding under Observation Attacks: A Principled Adversarial-Plus-Smoothing Training Recipe12 May 2026
- 2026When Actions Disappear: Adversarial Action Removal in Self-Play Reinforcement Learning4 May 2026
12 newest of 141 All 141 (JSON)