mlsanityData to 7 Oct 2026

Parameterized Decision-making with Multi-modal Perception for Autonomous Driving

arXiv 2312.11935 ↗First posted 19 Dec 2023cs.AIcited by 7

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Autonomous driving is an emerging technology that has advanced rapidly over the last decade. Modern transportation is expected to benefit greatly from a wise decision-making framework of autonomous vehicles, including the improvement of mobility and the minimization of risks and travel time. However, existing methods either ignore the complexity of environments only fitting straight roads, or ignore the impact on surrounding vehicles during optimization phases, leading to weak environmental adaptability and incomplete optimization objectives. To address these limitations, we propose a parameterized decision-making framework with multi-modal perception based on deep reinforcement learning, called AUTO. We conduct a comprehensive perception to capture the state features of various traffic participants around the autonomous vehicle, based on which we design a graph-based model to learn a state representation of the multi-modal semantic features. To distinguish between lane-following and lane-changing, we decompose an action of the autonomous vehicle into a parameterized action structure that first decides whether to change lanes and then computes an exact action to execute. A hybrid reward function takes into account aspects of safety, traffic efficiency, passenger comfort, and impact to guide the framework to generate optimal actions. In addition, we design a regularization term and a multi-worker paradigm to enhance the training. Extensive experiments offer evidence that AUTO can advance state-of-the-art in terms of both macroscopic and microscopic effectiveness.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 33 of its 79 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

33 of 79 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  8. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  9. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  10. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  11. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  12. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
21 more
  1. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 313
  2. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  3. 2018Distributed Prioritized Experience Replaycited by 260
  4. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  5. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  6. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  7. 2021Offline RL Without Off-Policy Evaluationcited by 150
  8. 2020Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehiclescited by 101
  9. 2015Deep Reinforcement Learning in Parameterized Action Spacecited by 65
  10. 2015Reinforcement Learning with Parameterized Actionscited by 61
  11. 2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 59
  12. 2019Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Drivingcited by 56
  13. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safetycited by 48
  14. 2020Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Surveycited by 48
  15. 2017Dissipation of stop-and-go waves via control of autonomous vehicles: Field experimentscited by 41
  16. 2019Social Attention for Autonomous Decision-Making in Dense Trafficcited by 39
  17. 2020Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learningcited by 27
  18. 2019Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraintscited by 24
  19. 2019Autonomous Highway Driving using Deep Reinforcement Learningcited by 22
  20. 2022Multi-task Safe Reinforcement Learning for Navigating Intersections in Dense Trafficcited by 7
  21. 2021Modeling Spatial Nonstationarity via Deformable Convolutions for Deep Traffic Flow Predictioncited by 4

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 3.

7 papers.

  1. 2024PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Drivingcited by 17
  2. 2024Enhancing Healthcare through Large Language Models: A Study on Medical Question Answeringcited by 14
  3. 2024Fine-Tuning Gemma-7B for Enhanced Sentiment Analysis of Financial News Headlinescited by 8
  4. 2024Advancing Prompt Recovery in NLP: A Deep Dive into the Integration of Gemma-2b-it and Phi2 Modelscited by 2
  5. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  6. 2025Multi-Timescale Hierarchical Reinforcement Learning for Unified Behavior and Control of Autonomous Drivingcited by 0
  7. 2025Energy-Efficient Autonomous Driving with Adaptive Perception and Robust Decisioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  2. 2025A Hybrid Input based Deep Reinforcement Learning for Lane Change Decision-Making of Autonomous Vehicle
  3. 2024Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach
  4. 2021A Versatile and Efficient Reinforcement Learning Framework for Autonomous Driving
  5. 2025Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving
  6. 2025An Intention-driven Lane Change Framework Considering Heterogeneous Dynamic Cooperation in Mixed-traffic Environment ↗
  7. 2021Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environment
  8. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions