# Model-Based Actor-Critic with Chance Constraint for Stochastic System

arXiv: https://arxiv.org/abs/2012.10716 · HTML: https://mlsanity.com/paper/2012.10716/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2020.

## Built on by (most cited first, 9 of 9)

- [Model-based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangian](https://mlsanity.com/paper/2108.11623/) (2021)
- [DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planning](https://mlsanity.com/paper/2406.09953/) (2024)
- [Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning](https://mlsanity.com/paper/2102.08539/) (2021)
- [EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model](https://mlsanity.com/paper/2210.00498/) (2022)
- [Improve Generalization of Driving Policy at Signalized Intersections with Adversarial Learning](https://mlsanity.com/paper/2204.04403/) (2022)
- [Learning Cooperative Oversubscription for Cloud by Chance-Constrained Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2211.11759/) (2022)
- [COIN: Chance-Constrained Imitation Learning for Uncertainty-aware Adaptive Resource Oversubscription Policy](https://mlsanity.com/paper/2401.07051/) (2024)
- [Predictive Lagrangian Optimization for Constrained Reinforcement Learning](https://mlsanity.com/paper/2501.15217/) (2025)
- [Learning Chance-Constrained MDPs with Bellman Distributional Certificates](https://mlsanity.com/paper/2609.30856/) (2026)

## Abstract (arXiv)

Safety is essential for reinforcement learning (RL) applied in real-world situations. Chance constraints are suitable to represent the safety requirements in stochastic systems. Previous chance-constrained RL methods usually have a low convergence rate, or only learn a conservative policy. In this paper, we propose a model-based chance constrained actor-critic (CCAC) algorithm which can efficiently learn a safe and non-conservative policy. Different from existing methods that optimize a conservative lower bound, CCAC directly solves the original chance constrained problems, where the objective function and safe probability is simultaneously optimized with adaptive weights. In order to improve the convergence rate, CCAC utilizes the gradient of dynamic model to accelerate policy optimization. The effectiveness of CCAC is demonstrated by a stochastic car-following task. Experiments indicate that compared with previous RL methods, CCAC improves the performance while guaranteeing safety, with a five times faster convergence rate. It also has 100 times higher online computation efficiency than traditional safety techniques such as stochastic model predictive control.