# Language-Driven Policy Distillation for Cooperative Driving in Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2410.24152 · HTML: https://mlsanity.com/paper/2410.24152/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 21 of 21)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation](https://mlsanity.com/paper/1708.05144/) (2017)
- [Grounding Language with Visual Affordances over Unstructured Data](https://mlsanity.com/paper/2210.01911/) (2022)
- [LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions](https://mlsanity.com/paper/2405.11106/) (2024)
- [Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration](https://mlsanity.com/paper/2405.14314/) (2024)
- [Large Language Models Are Semi-Parametric Reinforcement Learning Agents](https://mlsanity.com/paper/2306.07929/) (2023)
- [Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic](https://mlsanity.com/paper/2105.05701/) (2021)
- [AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learning](https://mlsanity.com/paper/2404.06345/) (2024)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents](https://mlsanity.com/paper/2311.13373/) (2023)
- [Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework](https://mlsanity.com/paper/2409.12812/) (2024)
- [Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach with Attention and Hierarchical Game Priors](https://mlsanity.com/paper/2409.05712/) (2024)
- [Cooperative Decision Making of Connected Automated Vehicles at Multi-lane Merging Zone: A Coalitional Game Approach](https://mlsanity.com/paper/2103.07887/) (2021)
- [Decision Making for Connected Automated Vehicles at Urban Intersections Considering Social and Individual Benefits](https://mlsanity.com/paper/2201.01428/) (2022)
- [Delay-Aware Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control with Model-based Stability Enhancement](https://mlsanity.com/paper/2404.15696/) (2024)
- [MAPPO-PIS: A Multi-Agent Proximal Policy Optimization Method with Prior Intent Sharing for CAVs' Cooperative Decision-Making](https://mlsanity.com/paper/2408.06656/) (2024)
- [Coordinating CAV Swarms at Intersections with a Deep Learning Model](https://mlsanity.com/paper/2211.05297/) (2022)

## Built on by (most cited first, 14 of 14)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [Multi-agent Embodied AI: Advances and Future Directions](https://mlsanity.com/paper/2505.05108/) (2025)
- [VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learning](https://mlsanity.com/paper/2506.09049/) (2025)
- [TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning](https://mlsanity.com/paper/2502.01387/) (2025)
- [AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent](https://mlsanity.com/paper/2602.03955/) (2026)
- [Towards Human-Centric Autonomous Driving: A Fast-Slow Architecture Integrating Large Language Model Guidance with Reinforcement Learning](https://mlsanity.com/paper/2505.06875/) (2025)
- [Distilling On-device Language Models for Robot Planning with Minimal Human Intervention](https://mlsanity.com/paper/2506.17486/) (2025)
- [GauDP: Reinventing Multi-Agent Collaboration through Gaussian-Image Synergy in Diffusion Policies](https://mlsanity.com/paper/2511.00998/) (2025)
- [Virtual Traffic Police: Large Language Model-Augmented Traffic Signal Control for Unforeseen Incidents](https://mlsanity.com/paper/2601.15816/) (2026)
- [LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2603.24361/) (2026)
- [Enforcing Cooperative Safety for Reinforcement Learning-based Mixed-Autonomy Platoon Control](https://mlsanity.com/paper/2411.10031/) (2024)
- [Game-Theoretic Risk-Shaped Reinforcement Learning for Safe Autonomous Driving](https://mlsanity.com/paper/2510.10960/) (2025)
- [Are LLMs The Way Forward? A Case Study on LLM-Guided Reinforcement Learning for Decentralized Autonomous Driving](https://mlsanity.com/paper/2511.12751/) (2025)
- [MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation](https://mlsanity.com/paper/2605.17743/) (2026)

## Abstract (arXiv)

The cooperative driving technology of Connected and Autonomous Vehicles (CAVs) is crucial for improving the efficiency and safety of transportation systems. Learning-based methods, such as Multi-Agent Reinforcement Learning (MARL), have demonstrated strong capabilities in cooperative decision-making tasks. However, existing MARL approaches still face challenges in terms of learning efficiency and performance. In recent years, Large Language Models (LLMs) have rapidly advanced and shown remarkable abilities in various sequential decision-making tasks. To enhance the learning capabilities of cooperative agents while ensuring decision-making efficiency and cost-effectiveness, we propose LDPD, a language-driven policy distillation method for guiding MARL exploration. In this framework, a teacher agent based on LLM trains smaller student agents to achieve cooperative decision-making through its own decision-making demonstrations. The teacher agent enhances the observation information of CAVs and utilizes LLMs to perform complex cooperative decision-making reasoning, which also leverages carefully designed decision-making tools to achieve expert-level decisions, providing high-quality teaching experiences. The student agent then refines the teacher's prior knowledge into its own model through gradient policy updates. The experiments demonstrate that the students can rapidly improve their capabilities with minimal guidance from the teacher and eventually surpass the teacher's performance. Extensive experiments show that our approach demonstrates better performance and learning efficiency compared to baseline methods.