mlsanityData to 14 Oct 2026

SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehicles

arXiv 2409.15105 ↗First posted 23 Sep 2024cs.AI cs.MA cs.SYcited by 1

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

In mixed autonomy traffic environment, every decision made by an autonomous-driving car may have a great impact on the transportation system. Because of the complex interaction between vehicles, it is challenging to make decisions that can ensure both high traffic efficiency and safety now and futher. Connected automated vehicles (CAVs) have great potential to improve the quality of decision-making in this continuous, highly dynamic and interactive environment because of their stronger sensing and communicating ability. For multi-vehicle collaborative decision-making algorithms based on deep reinforcement learning (DRL), we need to represent the interactions between vehicles to obtain interactive features. The representation in this aspect directly affects the learning efficiency and the quality of the learned policy. To this end, we propose a CAV decision-making architecture based on transformer and reinforcement learning algorithms. A learnable policy token is used as the learning medium of the multi-vehicle joint policy, the states of all vehicles in the area of interest can be adaptively noticed in order to extract interactive features among agents. We also design an intuitive physical positional encodings, the redundant location information of which optimizes the performance of the network. Simulations show that our model can make good use of all the state information of vehicles in traffic scenario, so as to obtain high-quality driving decisions that meet efficiency and safety objectives. The comparison shows that our method significantly improves existing DRL-based multi-vehicle cooperative decision-making algorithms.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 41 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

20 of 41 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2017Deep Interest Network for Click-Through Rate Predictioncited by 544
  6. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 420
  7. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  8. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  9. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 193
  10. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  11. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  12. 2019Encoding word order in complex embeddingscited by 53
8 more
  1. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Drivingcited by 24
  2. 2018Foundations of Sequence-to-Sequence Modeling for Time Seriescited by 15
  3. 20223DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformerscited by 15
  4. 2020A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Networkcited by 9
  5. 2022Holistic Transformer: A Joint Neural Network for Trajectory Prediction and Decision-Making of Autonomous Vehiclescited by 8
  6. 2022Decision Making for Connected Automated Vehicles at Urban Intersections Considering Social and Individual Benefitscited by 7
  7. 2022 T3 OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Areacited by 5
  8. 2023Bringing Diversity to Autonomous Vehicles: An Interpretable Multi-vehicle Decision-making and Planning Frameworkcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Qualitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic
  2. 2024Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Survey
  3. 2024Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach with Attention and Hierarchical Game Priors
  4. 2023Shared Information-Based Safe And Efficient Behavior Planning For Connected Autonomous Vehicles
  5. 2025BIDA: A Bi-level Interaction Decision-making Algorithm for Autonomous Vehicles in Dynamic Traffic Scenarios
  6. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning
  7. 2024Demystifying the Physics of Deep Reinforcement Learning-Based Autonomous Vehicle Decision-Making
  8. 2024Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach