# Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2102.07359 · HTML: https://mlsanity.com/paper/2102.07359/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 15 Feb 2021.

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [A Survey on Knowledge Graph-Based Recommender Systems](https://mlsanity.com/paper/2003.00911/) (2020)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1906.04737/) (2019)
- [Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1901.11454/) (2019)
- [Joint Air Quality and Weather Prediction Based on Multi-Adversarial Spatiotemporal Networks](https://mlsanity.com/paper/2012.15037/) (2020)
- [Deep Reinforcement Learning for Sponsored Search Real-time Bidding](https://mlsanity.com/paper/1803.00259/) (2018)
- [Semi-Supervised Hierarchical Recurrent Graph Neural Network for City-Wide Parking Availability Prediction](https://mlsanity.com/paper/1911.10516/) (2019)
- [Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching](https://mlsanity.com/paper/1910.02591/) (2019)
- [AutoFS: Automated Feature Selection via Diversity-aware Interactive Reinforcement Learning](https://mlsanity.com/paper/2008.12001/) (2020)
- [CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms](https://mlsanity.com/paper/1905.11353/) (2019)
- [Out-of-Town Recommendation with Travel Intention Modeling](https://mlsanity.com/paper/2101.12555/) (2021)

## Built on by (most cited first, 20 of 20)

- [Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis](https://mlsanity.com/paper/2109.03699/) (2021)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions](https://mlsanity.com/paper/2109.03540/) (2021)
- [MugRep: A Multi-Task Hierarchical Graph Representation Learning Framework for Real Estate Appraisal](https://mlsanity.com/paper/2107.05180/) (2021)
- [Irregular Traffic Time Series Forecasting Based on Asynchronous Spatio-Temporal Graph Convolutional Network](https://mlsanity.com/paper/2308.16818/) (2023)
- [Machine Learning for Urban Air Quality Analytics: A Survey](https://mlsanity.com/paper/2310.09620/) (2023)
- [Reinforcement Learning-enhanced Shared-account Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2206.08088/) (2022)
- [EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning](https://mlsanity.com/paper/2308.16008/) (2023)
- [Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey](https://mlsanity.com/paper/2108.04462/) (2021)
- [Feature and Instance Joint Selection: A Reinforcement Learning Perspective](https://mlsanity.com/paper/2205.07867/) (2022)
- [A Preference-aware Meta-optimization Framework for Personalized Vehicle Energy Consumption Estimation](https://mlsanity.com/paper/2306.14421/) (2023)
- [Reinforcement Learning-based Placement of Charging Stations in Urban Road Networks](https://mlsanity.com/paper/2206.06011/) (2022)
- [Coupling User Preference with External Rewards to Enable Driver-centered and Resource-aware EV Charging Recommendation](https://mlsanity.com/paper/2210.12693/) (2022)
- [OpenSiteRec: An Open Dataset for Site Recommendation](https://mlsanity.com/paper/2307.00856/) (2023)
- [A Communication-Efficient Decentralized Actor-Critic Algorithm](https://mlsanity.com/paper/2510.19199/) (2025)
- [Breaking the Filter Bubble: A Semantic Pareto-DQN Framework for Multi-Objective Recommendation](https://mlsanity.com/paper/2606.24042/) (2026)
- [SPAP: Simultaneous Demand Prediction and Planning for Electric Vehicle Chargers in a New City](https://mlsanity.com/paper/2110.09452/) (2021)
- [Agent-Based Decentralized Energy Management of EV Charging Station with Solar Photovoltaics via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2505.18750/) (2025)
- [Resilient Charging Infrastructure via Decentralized Coordination of Electric Vehicles at Scale](https://mlsanity.com/paper/2511.20943/) (2025)
- [Incentive-Aligned Vehicle-to-Vehicle Energy Trading via Nash-Integrated Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.22363/) (2026)

## Abstract (arXiv)

Electric Vehicle (EV) has become a preferable choice in the modern transportation system due to its environmental and energy sustainability. However, in many large cities, EV drivers often fail to find the proper spots for charging, because of the limited charging infrastructures and the spatiotemporally unbalanced charging demands. Indeed, the recent emergence of deep reinforcement learning provides great potential to improve the charging experience from various aspects over a long-term horizon. In this paper, we propose a framework, named Multi-Agent Spatio-Temporal Reinforcement Learning (Master), for intelligently recommending public accessible charging stations by jointly considering various long-term spatiotemporal factors. Specifically, by regarding each charging station as an individual agent, we formulate this problem as a multi-objective multi-agent reinforcement learning task. We first develop a multi-agent actor-critic framework with the centralized attentive critic to coordinate the recommendation between geo-distributed agents. Moreover, to quantify the influence of future potential charging competition, we introduce a delayed access strategy to exploit the knowledge of future charging competition during training. After that, to effectively optimize multiple learning objectives, we extend the centralized attentive critic to multi-critics and develop a dynamic gradient re-weighting strategy to adaptively guide the optimization direction. Finally, extensive experiments on two real-world datasets demonstrate that Master achieves the best comprehensive performance compared with nine baseline approaches.