# Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecification

arXiv: https://arxiv.org/abs/2010.10644 · HTML: https://mlsanity.com/paper/2010.10644/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 16 of 16)

- [COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation](https://mlsanity.com/paper/2204.08957/) (2022)
- [Robust Constrained Reinforcement Learning](https://mlsanity.com/paper/2209.06866/) (2022)
- [Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learning](https://mlsanity.com/paper/2301.12593/) (2023)
- [Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Form](https://mlsanity.com/paper/2408.16286/) (2024)
- [Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guarantees](https://mlsanity.com/paper/2301.13375/) (2023)
- [Sparse Adversarial Attack in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2205.09362/) (2022)
- [Constrained Reinforcement Learning Under Model Mismatch](https://mlsanity.com/paper/2405.01327/) (2024)
- [SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer](https://mlsanity.com/paper/2509.18648/) (2025)
- [C-MCTS: Safe Planning with Monte Carlo Tree Search](https://mlsanity.com/paper/2305.16209/) (2023)
- [Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guarantees](https://mlsanity.com/paper/2505.19238/) (2025)
- [Robust Lagrangian and Adversarial Policy Gradient for Robust Constrained Markov Decision Processes](https://mlsanity.com/paper/2308.11267/) (2023)
- [A Comprehensive Survey on Physical Risk Control in the Era of Foundation Model-enabled Robotics](https://mlsanity.com/paper/2505.12583/) (2025)
- [Primal-Only Actor Critic Algorithm for Robust Constrained Average Cost MDPs](https://mlsanity.com/paper/2511.05758/) (2025)
- [Provably Efficient Sample Complexity for Robust CMDP](https://mlsanity.com/paper/2511.07486/) (2025)
- [TCRL: Temporal-Coupled Adversarial Training for Robust Constrained Reinforcement Learning in Worst-Case Scenarios](https://mlsanity.com/paper/2602.13040/) (2026)
- [Optimistic Policy Learning under Pessimistic Adversaries with Regret and Violation Guarantees](https://mlsanity.com/paper/2604.14243/) (2026)

## Abstract (arXiv)

Many real-world physical control systems are required to satisfy constraints upon deployment. Furthermore, real-world systems are often subject to effects such as non-stationarity, wear-and-tear, uncalibrated sensors and so on. Such effects effectively perturb the system dynamics and can cause a policy trained successfully in one domain to perform poorly when deployed to a perturbed version of the same domain. This can affect a policy's ability to maximize future rewards as well as the extent to which it satisfies constraints. We refer to this as constrained model misspecification. We present an algorithm that mitigates this form of misspecification, and showcase its performance in multiple simulated Mujoco tasks from the Real World Reinforcement Learning (RWRL) suite.