# Centralized vs. Decentralized Multi-Agent Reinforcement Learning for Enhanced Control of Electric Vehicle Charging Networks

arXiv: https://arxiv.org/abs/2404.12520 · HTML: https://mlsanity.com/paper/2404.12520/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 18 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Optimal Reward Baseline for Gradient-Based Reinforcement Learning](https://mlsanity.com/paper/1301.2315/) (2013)
- [Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.04402/) (2021)
- [Settling the Variance of Multi-Agent Policy Gradients](https://mlsanity.com/paper/2108.08612/) (2021)
- [Smart Online Charging Algorithm for Electric Vehicles via Customized Actor-Critic Learning](https://mlsanity.com/paper/2106.00854/) (2021)
- [Dynamic Scheduling for Charging Electric Vehicles: A Priority Rule](https://mlsanity.com/paper/1602.00372/) (2016)
- [A Survey of Algorithms for Distributed Charging Control of Electric Vehicles in Smart Grid](https://mlsanity.com/paper/1911.06500/) (2019)
- [Federated Reinforcement Learning for Real-Time Electric Vehicle Charging and Discharging Control](https://mlsanity.com/paper/2210.01452/) (2022)
- [Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks](https://mlsanity.com/paper/2308.08792/) (2023)
- [An Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control](https://mlsanity.com/paper/2308.12921/) (2023)

## Abstract (arXiv)

The widespread adoption of electric vehicles (EVs) poses several challenges to power distribution networks and smart grid infrastructure due to the possibility of significantly increasing electricity demands, especially during peak hours. Furthermore, when EVs participate in demand-side management programs, charging expenses can be reduced by using optimal charging control policies that fully utilize real-time pricing schemes. However, devising optimal charging methods and control strategies for EVs is challenging due to various stochastic and uncertain environmental factors. Currently, most EV charging controllers operate based on a centralized model. In this paper, we introduce a novel approach for distributed and cooperative charging strategy using a Multi-Agent Reinforcement Learning (MARL) framework. Our method is built upon the Deep Deterministic Policy Gradient (DDPG) algorithm for a group of EVs in a residential community, where all EVs are connected to a shared transformer. This method, referred to as CTDE-DDPG, adopts a Centralized Training Decentralized Execution (CTDE) approach to establish cooperation between agents during the training phase, while ensuring a distributed and privacy-preserving operation during execution. We theoretically examine the performance of centralized and decentralized critics for the DDPG-based MARL implementation and demonstrate their trade-offs. Furthermore, we numerically explore the efficiency, scalability, and performance of centralized and decentralized critics. Our theoretical and numerical results indicate that, despite higher policy gradient variances and training complexity, the CTDE-DDPG framework significantly improves charging efficiency by reducing total variation by approximately %36 and charging cost by around %9.1 on average...