#  T3 OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Area

arXiv: https://arxiv.org/abs/2203.00183 · HTML: https://mlsanity.com/paper/2203.00183/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 1 Mar 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers](https://mlsanity.com/paper/2101.08001/) (2021)
- [Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimization](https://mlsanity.com/paper/2110.13827/) (2021)
- [Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning](https://mlsanity.com/paper/2010.08193/) (2020)
- [Multiple Pursuer Multiple Evader Differential Games](https://mlsanity.com/paper/1911.03806/) (2019)
- [A Multi-intersection Vehicular Cooperative Control based on End-Edge-Cloud Computing](https://mlsanity.com/paper/2012.00500/) (2020)

## Built on by (most cited first, 5 of 5)

- [A Value Based Parallel Update MCTS Method for Multi-Agent Cooperative Decision Making of Connected and Automated Vehicles](https://mlsanity.com/paper/2409.13783/) (2024)
- [Graded-Q Reinforcement Learning with Information-Enhanced State Encoder for Hierarchical Collaborative Multi-Vehicle Pursuit](https://mlsanity.com/paper/2210.13470/) (2022)
- [SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehicles](https://mlsanity.com/paper/2409.15105/) (2024)
- [Progression Cognition Reinforcement Learning with Prioritized Experience for Multi-Vehicle Pursuit](https://mlsanity.com/paper/2306.05016/) (2023)
- [GITSR: Graph Interaction Transformer-based Scene Representation for Multi Vehicle Collaborative Decision-making](https://mlsanity.com/paper/2411.01608/) (2024)

## Abstract (arXiv)

Smart Internet of Vehicles (IoVs) combined with Artificial Intelligence (AI) will contribute to vehicle decision-making in the Intelligent Transportation System (ITS). Multi-Vehicle Pursuit games (MVP), a multi-vehicle cooperative ability to capture mobile targets, is becoming a hot research topic gradually. Although there are some achievements in the field of MVP in the open space environment, the urban area brings complicated road structures and restricted moving spaces as challenges to the resolution of MVP games. We define an Observation-constrained MVP (OMVP) problem in this paper and propose a Transformer-based Time and Team Reinforcement Learning scheme ( T3 OMVP) to address the problem. First, a new multi-vehicle pursuit model is constructed based on decentralized partially observed Markov decision processes (Dec-POMDP) to instantiate this problem. Second, by introducing and modifying the transformer-based observation sequence, QMIX is redefined to adapt to the complicated road structure, restricted moving spaces and constrained observations, so as to control vehicles to pursue the target combining the vehicle's observations. Third, a multi-intersection urban environment is built to verify the proposed scheme. Extensive experimental results demonstrate that the proposed  T3 OMVP scheme achieves significant improvements relative to state-of-the-art QMIX approaches by 9.66%~106.25%. Code is available at https://github.com/pipihaiziguai/T3OMVP.