mlsanityData to 7 Oct 2026

Generalized Multi-Objective Reinforcement Learning with Envelope Updates in URLLC-enabled Vehicular Networks

arXiv 2405.11331 ↗First posted 18 May 2024cs.LG cs.AI cs.NIcited by 5

On the map: End-to-end autonomous driving → Cooperative and V2X driving

Abstract (arXiv)

We develop a novel multi-objective reinforcement learning (MORL) framework to jointly optimize wireless network selection and autonomous driving policies in a multi-band vehicular network operating on conventional sub-6GHz spectrum and Terahertz frequencies. The proposed framework is designed to 1. maximize the traffic flow and minimize collisions by controlling the vehicle's motion dynamics (i.e., speed and acceleration), and 2. enhance the ultra-reliable low-latency communication (URLLC) while minimizing handoffs (HOs). We cast this problem as a multi-objective Markov Decision Process (MOMDP) and develop solutions for both predefined and unknown preferences of the conflicting objectives. Specifically, we develop a novel envelope MORL solution which develops policies that address multiple objectives with unknown preferences to the agent. While this approach reduces reliance on scalar rewards, policy effectiveness varying with different preferences is a challenge. To address this, we apply a generalized version of the Bellman equation and optimize the convex envelope of multi-objective Q values to learn a unified parametric representation capable of generating optimal policies across all possible preference configurations. Following an initial learning phase, our agent can execute optimal policies under any specified preference or infer preferences from minimal data samples. Numerical results validate the efficacy of the envelope-based MORL solution and demonstrate interesting insights related to the inter-dependency of vehicle motion dynamics, HOs, and the communication data rate. The proposed policies enable autonomous vehicles (AVs) to adopt safe driving behaviors with improved connectivity.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving. Builds on 19 of its 61 references that are in the map. First posted on arXiv 18 May 2024.

Builds on

19 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Reinforcement Learning for Joint V2I Network Selection and Autonomous Driving Policies3 Aug

Other references · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  3. 2017Hindsight Experience Replaycited by 1,096
  4. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  5. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 166
  6. 2018Urban Driving with Multi-Objective Deep Reinforcement Learningcited by 30
  7. 2019Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environmentcited by 20
  8. 2022Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computingcited by 10
  9. 2021A Generative Car-following Model Conditioned On Driving Stylescited by 8
  10. 2023Multi-UAV Speed Control with Collision Avoidance and Handover-aware Cell Association: DRL with Action Branchingcited by 5
  11. 2023Detection and Imputation based Two-Stage Denoising Diffusion Power System Measurement Recovery under Cyber-Physical Uncertaintiescited by 5
  12. 2021Mobility-Aware Performance in Hybrid RF and Terahertz Wireless Networkscited by 4
6 more
  1. 2020Reinforcement Learning Based Vehicle-cell Association Algorithm for Highly Mobile Millimeter Wave Communicationcited by 3
  2. 2020Interference and Coverage Analysis in Coexisting RF and Dense TeraHertz Wireless Networkscited by 3
  3. 2023Optimization of Speed and Network Deployment for Reliable V2I Communication in the Presence of Handoffs and Interferencecited by 3
  4. 2024Non-orthogonal Age-Optimal Information Dissemination in Vehicular Networks: A Meta Multi-Objective Reinforcement Learning Approachcited by 3
  5. 2023Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuverscited by 2
  6. 2023A Tractable Handoff-aware Rate Outage Approximation with Applications to THz-enabled Vehicular Network Optimizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2025Hierarchical and Collaborative LLM-Based Control for Multi-UAV Motion and Communication in Integrated Terrestrial and Non-Terrestrial Networkscited by 5
  2. 2025CVaR-Based Variational Quantum Optimization for User Association in Handoff-Aware Vehicular Networkscited by 2
  3. 2024Hybrid LLM-DDQN based Joint Optimization of V2I Communication and Autonomous Drivingcited by 1
  4. 2026Hierarchical LLM-Driven Control for HAPS-Assisted UAV Networks: Joint Optimization of Flight and Connectivitycited by 0
  5. 2026Intelligent Multi-UAV Navigation in ITNTNs: A Hierarchical LLM Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Optimizing Vehicular Networks with Variational Quantum Circuits-based Reinforcement Learning
  2. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Driving
  3. 2026Offline Multi-Agent Reinforcement Learning for 6G Communications: Fundamentals, Applications and Future Directions
  4. 2026Multi-Objective Reinforcement Learning for Tactical Decision Making for Trucks in Highway Traffic
  5. 2025Preference-based Multi-Objective Reinforcement Learning
  6. 2025Offline Reinforcement Learning for Mobility Robustness Optimization
  7. 2026Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control
  8. 2022Scalable Model-based Policy Optimization for Decentralized Networked Systems