# A Robust and Constrained Multi-Agent Reinforcement Learning Electric Vehicle Rebalancing Method in AMoD Systems

arXiv: https://arxiv.org/abs/2209.08230 · HTML: https://mlsanity.com/paper/2209.08230/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 14 of 14)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems](https://mlsanity.com/paper/1906.00331/) (2019)
- [Policy Gradient Method For Robust Reinforcement Learning](https://mlsanity.com/paper/2205.07344/) (2022)
- [Deep Reinforcement Learning for Multi-Driver Vehicle Dispatching and Repositioning Problem](https://mlsanity.com/paper/1911.11260/) (2019)
- [Data-Driven Model Predictive Control of Autonomous Mobility-on-Demand Systems](https://mlsanity.com/paper/1709.07032/) (2017)
- [Definition and evaluation of model-free coordination of electrical vehicle charging with reinforcement learning](https://mlsanity.com/paper/1809.10679/) (2018)
- [Model Predictive Control of Autonomous Mobility-on-Demand Systems](https://mlsanity.com/paper/1509.03985/) (2015)
- [Predictive Positioning and Quality Of Service Ridesharing for Campus Mobility On Demand Systems](https://mlsanity.com/paper/1609.08116/) (2016)
- [Dynamic vehicle redistribution and online price incentives in shared mobility systems](https://mlsanity.com/paper/1304.3949/) (2013)
- [A BCMP Network Approach to Modeling and Controlling Autonomous Mobility-on-Demand Systems](https://mlsanity.com/paper/1607.04357/) (2016)

## Built on by (most cited first, 11 of 11)

- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems](https://mlsanity.com/paper/2312.08884/) (2023)
- [Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data](https://mlsanity.com/paper/2411.03810/) (2024)
- [Optimizing Irrigation Efficiency using Deep Reinforcement Learning in the Field](https://mlsanity.com/paper/2304.01435/) (2023)
- [Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization](https://mlsanity.com/paper/2310.09833/) (2023)
- [Towards Robust Vision Transformer via Masked Adaptive Ensemble](https://mlsanity.com/paper/2407.15385/) (2024)
- [CoDriveVLM: VLM-Enhanced Urban Cooperative Dispatching and Motion Planning for Future Autonomous Mobility on Demand Systems](https://mlsanity.com/paper/2501.06132/) (2025)
- [Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Duality](https://mlsanity.com/paper/2508.17448/) (2025)
- [Robo-taxi Fleet Coordination at Scale via Reinforcement Learning](https://mlsanity.com/paper/2504.06125/) (2025)
- [Multi-modal Spatial Clustering for Spatial Transcriptomics Utilizing High-resolution Histology Images](https://mlsanity.com/paper/2411.02534/) (2024)
- [Bridging Policy and Real-World Dynamics: LLM-Augmented Rebalancing for Shared Micromobility Systems](https://mlsanity.com/paper/2603.00176/) (2026)

## Abstract (arXiv)

Electric vehicles (EVs) play critical roles in autonomous mobility-on-demand (AMoD) systems, but their unique charging patterns increase the model uncertainties in AMoD systems (e.g. state transition probability). Since there usually exists a mismatch between the training and test/true environments, incorporating model uncertainty into system design is of critical importance in real-world applications. However, model uncertainties have not been considered explicitly in EV AMoD system rebalancing by existing literature yet, and the coexistence of model uncertainties and constraints that the decision should satisfy makes the problem even more challenging. In this work, we design a robust and constrained multi-agent reinforcement learning (MARL) framework with state transition kernel uncertainty for EV AMoD systems. We then propose a robust and constrained MARL algorithm (ROCOMA) with robust natural policy gradients (RNPG) that trains a robust EV rebalancing policy to balance the supply-demand ratio and the charging utilization rate across the city under model uncertainty. Experiments show that the ROCOMA can learn an effective and robust rebalancing policy. It outperforms non-robust MARL methods in the presence of model uncertainties. It increases the system fairness by 19.6% and decreases the rebalancing costs by 75.8%.