# Mix Q-learning for Lane Changing: A Collaborative Decision-Making Method in Multi-Agent Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2406.09755 · HTML: https://mlsanity.com/paper/2406.09755/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 54 references that are in the map.
- First posted on arXiv 14 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020)
- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints](https://mlsanity.com/paper/1904.00231/) (2019)
- [Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment](https://mlsanity.com/paper/1909.11538/) (2019)
- [Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic](https://mlsanity.com/paper/2111.06318/) (2021)
- [CMetric: A Driving Behavior Measure Using Centrality Functions](https://mlsanity.com/paper/2003.04424/) (2020)
- [B-GAP: Behavior-Rich Simulation and Navigation for Autonomous Driving](https://mlsanity.com/paper/2011.03748/) (2020)
- [Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Model](https://mlsanity.com/paper/1910.04803/) (2019)
- [Parameter Sharing Deep Deterministic Policy Gradient for Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1710.00336/) (2017)
- [Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization](https://mlsanity.com/paper/1909.10651/) (2019)
- [A Learning-based Discretionary Lane-Change Decision-Making Model with Driving Style Awareness](https://mlsanity.com/paper/2010.09533/) (2020)
- [Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks](https://mlsanity.com/paper/1609.02993/) (2016)

## Built on by (most cited first, 1 of 1)

- [PALCAS: A Priority-Aware Intelligent Lane Change Advisory System for Autonomous Vehicles using Federated Reinforcement Learning](https://mlsanity.com/paper/2604.27118/) (2026)

## Abstract (arXiv)

Lane-changing decisions, which are crucial for autonomous vehicle path planning, face practical challenges due to rule-based constraints and limited data. Deep reinforcement learning has become a major research focus due to its advantages in data acquisition and interpretability. However, current models often overlook collaboration, which affects not only impacts overall traffic efficiency but also hinders the vehicle's own normal driving in the long run. To address the aforementioned issue, this paper proposes a method named Mix Q-learning for Lane Changing(MQLC) that integrates a hybrid value Q network, taking into account both collective and individual benefits for the greater good. At the collective level, our method coordinates the individual Q and global Q networks by utilizing global information. This enables agents to effectively balance their individual interests with the collective benefit. At the individual level, we integrated a deep learning-based intent recognition module into our observation and enhanced the decision network. These changes provide agents with richer decision information and more accurate feature extraction for improved lane-changing decisions. This strategy enables the multi-agent system to learn and formulate optimal decision-making strategies effectively. Our MQLC model, through extensive experimental results, impressively outperforms other state-of-the-art multi-agent decision-making methods, achieving significantly safer and faster lane-changing decisions. The code is available at https:github.com/pku-smart-city/source_code/tree/main/MQLC.