# Reinforcement Learning with Distributed MPC for Fuel-Efficient Platoon Control with Discrete Gear Transitions

arXiv: https://arxiv.org/abs/2601.18294 · HTML: https://mlsanity.com/paper/2601.18294/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 46 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 11 of 11)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Communication-Efficient Decentralized Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control](https://mlsanity.com/paper/2308.02345/) (2023)
- [Distributed Model Predictive Control for Heterogeneous Vehicle Platoons under Unidirectional Topologies](https://mlsanity.com/paper/1603.03225/) (2016)
- [Semi-Definite Relaxation Based ADMM for Cooperative Planning and Control of Connected Autonomous Vehicles](https://mlsanity.com/paper/2101.00201/) (2021)
- [A Safe Control Architecture Based on a Model Predictive Control Supervisor for Autonomous Driving](https://mlsanity.com/paper/2201.13298/) (2022)
- [Learning-Based MPC for Fuel Efficient Control of Autonomous Vehicles with Discrete Gear Selection](https://mlsanity.com/paper/2503.11359/) (2025)
- [Constraint-Driven Optimal Control of Multi-Agent Systems: A Highway Platooning Case Study](https://mlsanity.com/paper/2109.05988/) (2021)
- [A Comparison Benchmark for Distributed Hybrid MPC Control Methods: Distributed Vehicle Platooning](https://mlsanity.com/paper/2401.09878/) (2024)
- [Integrating Reinforcement Learning and Model Predictive Control with Applications to Microgrids](https://mlsanity.com/paper/2409.11267/) (2024)
- [Learning-Based Model Predictive Control for Piecewise Affine Systems with Feasibility Guarantees](https://mlsanity.com/paper/2412.00490/) (2024)

## Abstract (arXiv)

Cooperative control of groups of autonomous vehicles (AVs), i.e., platoons, is a promising direction to improving the efficiency of autonomous transportation systems. In this context, distributed co-optimization of both vehicle speed and gear position can offer benefits for fuel-efficient driving. To this end, model predictive control (MPC) is a popular approach, optimizing the speed and gear-shift schedule while explicitly considering the vehicles' dynamics over a prediction window. However, optimization over both the vehicles' continuous dynamics and discrete gear positions is computationally intensive, and may require overly long sample times or high-end hardware for real-time implementation. This work proposes a reinforcement learning (RL)-based distributed MPC approach to address this issue. For each vehicle in the platoon, a policy is trained to select and fix the gear positions across the prediction window of a local MPC controller, leaving a significantly simpler continuous optimization problem to be solved as part of a distributed MPC scheme. In order to reduce the computational cost of training and facilitate the scalability of the proposed approach to large platoons, the policies are parameterized such that the emergent multi-agent RL problem can be decoupled into single-agent learning tasks. In addition, a recurrent neural-network (RNN) architecture is proposed for the gear selection policy, such that the learning is scalable even as the number of possible gear-shift schedules grows exponentially with the MPC prediction horizon. In highway-driving simulations, the proposed approach is shown to have a significantly lower computation burden and a comparable performance in terms of fuel-efficient platoon control, with respect to pure MPC-based co-optimization.