# AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging

arXiv: https://arxiv.org/abs/2408.03624 · HTML: https://mlsanity.com/paper/2408.03624/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 78 references that are in the map.
- First posted on arXiv 7 Aug 2024.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning](https://mlsanity.com/paper/2207.07601/) (2022)
- [GPT-Driver: Learning to Drive with GPT](https://mlsanity.com/paper/2310.01415/) (2023)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time](https://mlsanity.com/paper/2310.17157/) (2023)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [Building Cooperative Embodied Agents Modularly with Large Language Models](https://mlsanity.com/paper/2307.02485/) (2023)
- [Convolutional Social Pooling for Vehicle Trajectory Prediction](https://mlsanity.com/paper/1805.06771/) (2018)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [A Language Agent for Autonomous Driving](https://mlsanity.com/paper/2311.10813/) (2023)
- [VLP: Vision Language Planning for Autonomous Driving](https://mlsanity.com/paper/2401.05577/) (2024)

## Built on by (most cited first, 22 of 22)

- [Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances](https://mlsanity.com/paper/2502.16804/) (2025)
- [IC3M: In-Car Multimodal Multi-object Monitoring for Abnormal Status of Both Driver and Passengers](https://mlsanity.com/paper/2410.02592/) (2024)
- [Plurals: A System for Guiding LLMs Via Simulated Social Ensembles](https://mlsanity.com/paper/2409.17213/) (2024)
- [Directed-CP: Directed Collaborative Perception for Connected and Autonomous Vehicles via Proactive Attention](https://mlsanity.com/paper/2409.08840/) (2024)
- [CP-Guard: Malicious Agent Detection and Defense in Collaborative Bird's Eye View Perception](https://mlsanity.com/paper/2412.12000/) (2024)
- [Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring](https://mlsanity.com/paper/2508.09085/) (2025)
- [GCP: Guarded Collaborative Perception with Spatial-Temporal Aware Malicious Agent Detection](https://mlsanity.com/paper/2501.02450/) (2025)
- [State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning](https://mlsanity.com/paper/2503.20613/) (2025)
- [Robust Deep Reinforcement Learning in Robotics via Adaptive Gradient-Masked Adversarial Attacks](https://mlsanity.com/paper/2503.20844/) (2025)
- [SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference](https://mlsanity.com/paper/2507.06567/) (2025)
- [Distribution-Aligned Decoding for Efficient LLM Task Adaptation](https://mlsanity.com/paper/2509.15888/) (2025)
- [Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security](https://mlsanity.com/paper/2605.23989/) (2026)
- [Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneity](https://mlsanity.com/paper/2603.21276/) (2026)
- [Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance](https://mlsanity.com/paper/2508.01057/) (2025)
- [Task-Aware Parameter-Efficient Fine-Tuning of Large Pre-Trained Models at the Edge](https://mlsanity.com/paper/2504.03718/) (2025)
- [SFL-LEO: Asynchronous Split-Federated Learning Design for LEO Satellite-Ground Network Framework](https://mlsanity.com/paper/2504.13479/) (2025)
- [Agentic Satellite-Augmented Low-Altitude Economy and Terrestrial Networks: A Survey on Generative Approaches](https://mlsanity.com/paper/2507.14633/) (2025)
- [CP-uniGuard: A Unified, Probability-Agnostic, and Adaptive Framework for Malicious Agent Detection and Defense in Multi-Agent Embodied Perception Systems](https://mlsanity.com/paper/2506.22890/) (2025)
- [Unified Context Evolution for LLM Agents](https://mlsanity.com/paper/2606.02304/) (2026)
- [DP-TRAE: A Dual-Phase Merging Transferable Reversible Adversarial Example for Image Privacy Protection](https://mlsanity.com/paper/2505.06860/) (2025)
- [Semore: VLM-guided Enhanced Semantic Motion Representations for Visual Reinforcement Learning](https://mlsanity.com/paper/2512.05172/) (2025)
- [DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization](https://mlsanity.com/paper/2601.19394/) (2026)

## Abstract (arXiv)

Ramp merging is one of the bottlenecks in traffic systems, which commonly cause traffic congestion, accidents, and severe carbon emissions. In order to address this essential issue and enhance the safety and efficiency of connected and autonomous vehicles (CAVs) at multi-lane merging zones, we propose a novel collaborative decision-making framework, named AgentsCoMerge, to leverage large language models (LLMs). Specifically, we first design a scene observation and understanding module to allow an agent to capture the traffic environment. Then we propose a hierarchical planning module to enable the agent to make decisions and plan trajectories based on the observation and the agent's own state. In addition, in order to facilitate collaboration among multiple agents, we introduce a communication module to enable the surrounding agents to exchange necessary information and coordinate their actions. Finally, we develop a reinforcement reflection guided training paradigm to further enhance the decision-making capability of the framework. Extensive experiments are conducted to evaluate the performance of our proposed method, demonstrating its superior efficiency and effectiveness for multi-agent collaborative decision-making under various ramp merging scenarios.