# Trust-MARL: Trust-Based Multi-Agent Reinforcement Learning Framework for Cooperative On-Ramp Merging Control in Heterogeneous Traffic Flow

arXiv: https://arxiv.org/abs/2506.12600 · HTML: https://mlsanity.com/paper/2506.12600/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 59 references that are in the map.
- First posted on arXiv 14 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies](https://mlsanity.com/paper/2304.10573/) (2023)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020)
- [Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic](https://mlsanity.com/paper/2105.05701/) (2021)
- [Multi-Agent Actor-Critic with Hierarchical Graph Attention Network](https://mlsanity.com/paper/1909.12557/) (2019)
- [RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Driving](https://mlsanity.com/paper/2503.13861/) (2025)
- [Randomized Value Functions via Multiplicative Normalizing Flows](https://mlsanity.com/paper/1806.02315/) (2018)
- [Driving Decision and Control for Autonomous Lane Change based on Deep Reinforcement Learning](https://mlsanity.com/paper/1904.10171/) (2019)
- [Multi-Task Trust Transfer for Human-Robot Interaction](https://mlsanity.com/paper/1807.01866/) (2018)
- [Cooperative Driving of Connected Autonomous Vehicles in Heterogeneous Mixed Traffic: A Game Theoretic Approach](https://mlsanity.com/paper/2305.03563/) (2023)
- [Characteristics Analysis of Autonomous Vehicle Pre-crash Scenarios](https://mlsanity.com/paper/2502.20789/) (2025)
- [Trust-based Consensus in Multi-Agent Reinforcement Learning Systems](https://mlsanity.com/paper/2205.12880/) (2022)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.06236/) (2023)
- [Trust-Aware Motion Planning for Human-Robot Collaboration under Distribution Temporal Logic Specifications](https://mlsanity.com/paper/2310.01163/) (2023)
- [Trust-Aware Resilient Control and Coordination of Connected and Automated Vehicles](https://mlsanity.com/paper/2305.16818/) (2023)
- [Learning-based social coordination to improve safety and robustness of cooperative autonomous vehicles in mixed traffic](https://mlsanity.com/paper/2211.11963/) (2022)
- [Safe Reinforcement Learning-Based Eco-Driving Control for Mixed Traffic Flows With Disturbances](https://mlsanity.com/paper/2401.17837/) (2024)
- [SECRM-2D: RL-Based Efficient and Comfortable Route-Following Autonomous Driving with Analytic Safety Guarantees](https://mlsanity.com/paper/2407.16857/) (2024)

## Built on by (most cited first, 3 of 3)

- [TGLD: A Trust-Aware Game-Theoretic Lane-Changing Decision Framework for Automated Vehicles in Heterogeneous Traffic](https://mlsanity.com/paper/2507.10075/) (2025)
- [Fault-Tolerant MARL for CAVs under Observation Perturbations for Highway On-Ramp Merging](https://mlsanity.com/paper/2511.23193/) (2025)
- [LLM-MLFFN: Multi-Level Autonomous Driving Behavior Feature Fusion via Large Language Model](https://mlsanity.com/paper/2603.02528/) (2026)

## Abstract (arXiv)

Intelligent transportation systems require connected and automated vehicles (CAVs) to conduct safe and efficient cooperation with human-driven vehicles (HVs) in complex real-world traffic environments. However, the inherent unpredictability of human behaviour, especially at bottlenecks such as highway on-ramp merging areas, often disrupts traffic flow and compromises system performance. To address the challenge of cooperative on-ramp merging in heterogeneous traffic environments, this study proposes a trust-based multi-agent reinforcement learning (Trust-MARL) framework. At the macro level, Trust-MARL enhances global traffic efficiency by leveraging inter-agent trust to improve bottleneck throughput and mitigate traffic shockwave through emergent group-level coordination. At the micro level, a dynamic trust mechanism is designed to enable CAVs to adjust their cooperative strategies in response to real-time behaviors and historical interactions with both HVs and other CAVs. Furthermore, a trust-triggered game-theoretic decision-making module is integrated to guide each CAV in adapting its cooperation factor and executing context-aware lane-changing decisions under safety, comfort, and efficiency constraints. An extensive set of ablation studies and comparative experiments validates the effectiveness of the proposed Trust-MARL approach, demonstrating significant improvements in safety, efficiency, comfort, and adaptability across varying CAV penetration rates and traffic densities.