Optimizing Electric Bus Charging Scheduling with Uncertainties Using Hierarchical Deep Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
The growing adoption of Electric Buses (EBs) represents a significant step toward sustainable development. By utilizing Internet of Things (IoT) systems, charging stations can autonomously determine charging schedules based on real-time data. However, optimizing EB charging schedules remains a critical challenge due to uncertainties in travel time, energy consumption, and fluctuating electricity prices. Moreover, to address real-world complexities, charging policies must make decisions efficiently across multiple time scales and remain scalable for large EB fleets. In this paper, we propose a Hierarchical Deep Reinforcement Learning (HDRL) approach that reformulates the original Markov Decision Process (MDP) into two augmented MDPs. To solve these MDPs and enable multi-timescale decision-making, we introduce a novel HDRL algorithm, namely Double Actor-Critic Multi-Agent Proximal Policy Optimization Enhancement (DAC-MAPPO-E). Scalability challenges of the Double Actor-Critic (DAC) algorithm for large-scale EB fleets are addressed through enhancements at both decision levels. At the high level, we redesign the decentralized actor network and integrate an attention mechanism to extract relevant global state information for each EB, decreasing the size of neural networks. At the low level, the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm is incorporated into the DAC framework, enabling decentralized and coordinated charging power decisions, reducing computational complexity and enhancing convergence speed. Extensive experiments with real-world data demonstrate the superior performance and scalability of DAC-MAPPO-E in optimizing EB fleet charging schedules.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 46 references that are in the map. First posted on arXiv 15 May 2025.
Builds on
7 of 46 references.
References · 7
- 2017Attention Is All You Needcited by 60,361
- 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
- 2016The Option-Critic Architecturecited by 448
- 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
- 2018An Adaptive Clipping Approach for Proximal Policy Optimizationcited by 14
- 2021Learning to Operate an Electric Vehicle Charging Station Considering Vehicle-grid Integrationcited by 7
- 2023Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learningcited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Electric Bus Charging Schedules Relying on Real Data-Driven Targets Based on Hierarchical Deep Reinforcement Learning
- 2025Optimizing Electric Vehicle Charging Station Placement Using Reinforcement Learning and Agent-Based Simulations
- 2025Reinforcement Learning for Hybrid Charging Stations Planning and Operation Considering Fixed and Mobile Chargers
- 2026Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches
- 2025Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards
- 2024Advancements in Electric Vehicle Charging Optimization: A Survey of Reinforcement Learning Approaches
- 2026LLM-Enhanced Multi-Agent Reinforcement Learning for Unified Electric Vehicles-Charging Station-Grid Optimization in Public Charging Systems
- 2025Physics-Informed Reinforcement Learning for Large-Scale EV Smart Charging Considering Distribution Network Voltage Constraints