# An Automated Reinforcement Learning Reward Design Framework with Large Language Model for Cooperative Platoon Coordination

arXiv: https://arxiv.org/abs/2504.19480 · HTML: https://mlsanity.com/paper/2504.19480/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 77 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Builds on (most cited first, 30 of 33)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [Text2Reward: Reward Shaping with Language Models for Reinforcement Learning](https://mlsanity.com/paper/2309.11489/) (2023)
- [Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions](https://mlsanity.com/paper/2203.15103/) (2022)
- [Learning Navigation Behaviors End-to-End with AutoRL](https://mlsanity.com/paper/1809.10124/) (2018)
- [Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity](https://mlsanity.com/paper/2210.09579/) (2022)
- [Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft](https://mlsanity.com/paper/2312.09238/) (2023)
- [Emergent Coordination Through Competition](https://mlsanity.com/paper/1902.07151/) (2019)
- [Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications](https://mlsanity.com/paper/2408.10215/) (2024)
- [Evolving Rewards to Automate Reinforcement Learning](https://mlsanity.com/paper/1905.07628/) (2019)
- [A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning](https://mlsanity.com/paper/2410.14660/) (2024)
- [Coordinating the Crowd: Inducing Desirable Equilibria in Non-Cooperative Systems](https://mlsanity.com/paper/1901.10923/) (2019)
- [Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2002.06723/) (2020)
- [Generating and Evolving Reward Functions for Highway Driving with Large Language Models](https://mlsanity.com/paper/2406.10540/) (2024)
- [Strategic Hub-Based Platoon Coordination under Uncertain Travel Times](https://mlsanity.com/paper/2104.00255/) (2021)
- [Optimal Coordination of Platoons of Connected and Automated Vehicles at Signal-Free Intersections](https://mlsanity.com/paper/2001.04866/) (2020)
- [Optimizing Two-Truck Platooning with Deadlines](https://mlsanity.com/paper/2210.01889/) (2022)
- [An Approximate Dynamic Programming Approach to Vehicle Platooning Coordination in Networks](https://mlsanity.com/paper/2308.03966/) (2023)
- [Fuel-Optimal Centralized Coordination of Truck Platooning Based on Shortest Paths](https://mlsanity.com/paper/1510.06603/) (2015)
- [Fuel-Efficient En Route Formation of Truck Platoons](https://mlsanity.com/paper/1704.08836/) (2017)
- [Efficient Dynamic Programming Solution to a Platoon Coordination Merge Problem With Stochastic Travel Times](https://mlsanity.com/paper/1705.04222/) (2017)
- [Multi-Fleet Platoon Matching: A Game-Theoretic Approach](https://mlsanity.com/paper/2202.08748/) (2022)
- [Event-Triggered Distributed Model Predictive Control for Platoon Coordination at Hubs in a Transport System](https://mlsanity.com/paper/2202.09105/) (2022)

## Built on by (most cited first, 1 of 1)

- [Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Development](https://mlsanity.com/paper/2510.12253/) (2025)

## Abstract (arXiv)

Reinforcement Learning (RL) has demonstrated excellent decision-making potential in platoon coordination problems. However, due to the variability of coordination goals, the complexity of the decision problem, and the time-consumption of trial-and-error in manual design, finding a well performance reward function to guide RL training to solve complex platoon coordination problems remains challenging. In this paper, we formally define the Platoon Coordination Reward Design Problem (PCRDP), extending the RL-based cooperative platoon coordination problem to incorporate automated reward function generation. To address PCRDP, we propose a Large Language Model (LLM)-based Platoon coordination Reward Design (PCRD) framework, which systematically automates reward function discovery through LLM-driven initialization and iterative optimization. In this method, LLM first initializes reward functions based on environment code and task requirements with an Analysis and Initial Reward (AIR) module, and then iteratively optimizes them based on training feedback with an evolutionary module. The AIR module guides LLM to deepen their understanding of code and tasks through a chain of thought, effectively mitigating hallucination risks in code generation. The evolutionary module fine-tunes and reconstructs the reward function, achieving a balance between exploration diversity and convergence stability for training. To validate our approach, we establish six challenging coordination scenarios with varying complexity levels within the Yangtze River Delta transportation network simulation. Comparative experimental results demonstrate that RL agents utilizing PCRD-generated reward functions consistently outperform human-engineered reward functions, achieving an average of 10% higher performance metrics in all scenarios.