mlsanityData to 7 Oct 2026

Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving

arXiv 2208.12263 ↗First posted 24 Aug 2022cs.LG cs.AI cs.ROcited by 24

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Decision-making for urban autonomous driving is challenging due to the stochastic nature of interactive traffic participants and the complexity of road structures. Although reinforcement learning (RL)-based decision-making scheme is promising to handle urban driving scenarios, it suffers from low sample efficiency and poor adaptability. In this paper, we propose Scene-Rep Transformer to improve the RL decision-making capabilities with better scene representation encoding and sequential predictive latent distillation. Specifically, a multi-stage Transformer (MST) encoder is constructed to model not only the interaction awareness between the ego vehicle and its neighbors but also intention awareness between the agents and their candidate routes. A sequential latent Transformer (SLT) with self-supervised learning objectives is employed to distill the future predictive information into the latent scene representation, in order to reduce the exploration space and speed up training. The final decision-making module based on soft actor-critic (SAC) takes as input the refined latent scene representation from the Scene-Rep Transformer and outputs driving actions. The framework is validated in five challenging simulated urban scenarios with dense traffic, and its performance is manifested quantitatively by the substantial improvements in data efficiency and performance in terms of success rate, safety, and efficiency. The qualitative results reveal that our framework is able to extract the intentions of neighbor agents to help make decisions and deliver more diversified driving behaviors.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 40 of its 74 references that are in the map. First posted on arXiv 24 Aug 2022.

Builds on

40 of 74 references.

Closest ancestors · 5

Same direction, newest first

  1. 2021DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networks11 Aug
  2. 2021Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving15 Apr
  3. 2021Increasing the Efficiency of Policy Learning for Autonomous Vehicles by Multi-Task Representation Learning26 Mar
  4. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving19 Mar
  5. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving18 Feb

Other references · 35

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  8. 2016End to End Learning for Self-Driving Carscited by 1,000
  9. 2016Deep Variational Information Bottleneckcited by 915
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  11. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
23 more
  1. 2019On Variational Bounds of Mutual Informationcited by 550
  2. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  3. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  4. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  5. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  6. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  7. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  8. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  9. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  10. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  11. 2020Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehiclescited by 101
  12. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  13. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  14. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  15. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safetycited by 48
  16. 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learningcited by 46
  17. 2020Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understandingcited by 42
  18. 2021MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Drivingcited by 41
  19. 2022Representation Uncertainty in Self-Supervised Learning as Variational Inferencecited by 22
  20. 2020Learning hierarchical behavior and motion planning for autonomous drivingcited by 15
  21. 2019Dynamic Interaction-Aware Scene Understanding for Reinforcement Learning in Autonomous Drivingcited by 14
  22. 2020Action-Based Representation Learning for Autonomous Drivingcited by 11
  23. 2020Learning predictive representations in autonomous driving to improve deep reinforcement learningcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 7; 2024: 9; 2025: 7.

24 papers.

Sort:

By area:

  1. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Drivingcited by 34
  2. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  3. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  4. 2024Hybrid-Prediction Integrated Planning for Autonomous Drivingcited by 18
  5. 2023Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Drivingcited by 16
  6. 2023Transformers in Reinforcement Learning: A Surveycited by 11
  7. 2023Transformer-Based Autonomous Driving Models and Deployment-Oriented Compression: A Surveycited by 8
  8. 2023Occupancy Prediction-Guided Neural Planner for Autonomous Drivingcited by 8
  9. 2025MambaFlow: A Novel and Flow-guided State Space Model for Scene Flow Estimationcited by 8
  10. 2024Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrationscited by 7
  11. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Drivingcited by 7
  12. 2023Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Drivingcited by 6
12 more
  1. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  2. 2023InteractionNet: Joint Planning and Prediction for Autonomous Driving with Transformerscited by 4
  3. 2024A Survey of Vision Transformers in Autonomous Driving: Current Trends and Future Directionscited by 4
  4. 2024A Safe and Efficient Self-evolving Algorithm for Decision-making and Control of Autonomous Driving Systemscited by 2
  5. 2024Deep Attention Driven Reinforcement Learning (DAD-RL) for Autonomous Decision-Making in Dynamic Environmentcited by 1
  6. 2024SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehiclescited by 1
  7. 2024Broad Critic Deep Actor Reinforcement Learning for Continuous Controlcited by 1
  8. 2025Offline Reinforcement Learning using Human-Aligned Reward Labeling for Autonomous Emergency Braking in Occluded Pedestrian Crossingcited by 1
  9. 2024GITSR: Graph Interaction Transformer-based Scene Representation for Multi Vehicle Collaborative Decision-makingcited by 0
  10. 2025Learning Geometrically-Informed Lyapunov Functions with Deep Diffeomorphic RBF Networkscited by 0
  11. 2025Resolve Highway Conflict in Multi-Autonomous Vehicle Controls with Local State Attentioncited by 0
  12. 2025Policy Disruption in Reinforcement Learning:Adversarial Attack with Large Language Models and Critical State Identificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving
  2. 2021Vision Transformer for Learning Driving Policies in Complex Multi-Agent Environments
  3. 2023RLAD: Reinforcement Learning from Pixels for Autonomous Driving in Urban Environments
  4. 2024From Imitation to Exploration: End-to-end Autonomous Driving based on World Model
  5. 2026MTA-RL: Robust Urban Driving via Multi-modal Transformer-based 3D Affordances and Reinforcement Learning
  6. 2021Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-lane Scenarios
  7. 2024An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving
  8. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation