mlsanityData to 14 Oct 2026

PeRP: Personalized Residual Policies For Congestion Mitigation Through Co-operative Advisory Systems

arXiv 2308.00864 ↗First posted 1 Aug 2023cs.LG cs.AI cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

Intelligent driving systems can be used to mitigate congestion through simple actions, thus improving many socioeconomic factors such as commute time and gas costs. However, these systems assume precise control over autonomous vehicle fleets, and are hence limited in practice as they fail to account for uncertainty in human behavior. Piecewise Constant (PC) Policies address these issues by structurally modeling the likeness of human driving to reduce traffic congestion in dense scenarios to provide action advice to be followed by human drivers. However, PC policies assume that all drivers behave similarly. To this end, we develop a co-operative advisory system based on PC policies with a novel driver trait conditioned Personalized Residual Policy, PeRP. PeRP advises drivers to behave in ways that mitigate traffic congestion. We first infer the driver's intrinsic traits on how they follow instructions in an unsupervised manner with a variational autoencoder. Then, a policy conditioned on the inferred trait adapts the action of the PC policy to provide the driver with a personalized recommendation. Our system is trained in simulation with novel driver modeling of instruction adherence. We show that our approach successfully mitigates congestion while adapting to different driver behaviors, with 4 to 22% improvement in average speed over baselines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 52 references that are in the map. First posted on arXiv 1 Aug 2023.

Builds on

26 of 52 references.

References · 26

  1. 2013Auto-Encoding Variational Bayescited by 4,053
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  5. 2018Residual Policy Learningcited by 164
  6. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  7. 2018Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysiscited by 110
  8. 2020Learning Latent Representations to Influence Multi-Agent Interactioncited by 93
  9. 2017Flow: A Modular Learning Framework for Mixed Autonomy Trafficcited by 73
  10. 2018How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Predictioncited by 46
  11. 2017Dissipation of stop-and-go waves via control of autonomous vehicles: Field experimentscited by 45
  12. 2020Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationshipscited by 37
14 more
  1. 2018Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncingcited by 36
  2. 2020Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learningcited by 28
  3. 2020Residual Policy Learning for Shared Autonomycited by 26
  4. 2021Influencing Towards Stable Multi-Agent Interactionscited by 24
  5. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 24
  6. 2017Simultaneous Policy Learning and Latent State Inference for Imitating Driver Behaviorcited by 16
  7. 2021Reinforcement Learning for Mixed Autonomy Intersectionscited by 16
  8. 2020Long-term Pedestrian Trajectory Prediction using Mutable Intention Filter and Warp LSTMcited by 10
  9. 2020A Taxonomy and Review of Algorithms for Modeling and Predicting Human Driver Behaviorcited by 9
  10. 2021Learning to Navigate Intersections with Unsupervised Driver Trait Inferencecited by 8
  11. 2022RILI: Robustly Influencing Latent Intentcited by 8
  12. 2018Data-Augmented Contact Model for Rigid Body Simulationcited by 6
  13. 2022Meta-path Analysis on Spatio-Temporal Graphs for Pedestrian Trajectory Predictioncited by 4
  14. 2023Towards Co-operative Congestion Mitigationcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3.

4 papers.

  1. 2024Model-Based Transfer Learning for Contextual Reinforcement Learningcited by 7
  2. 2023Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomycited by 6
  3. 2024Cooperative Advisory Residual Policies for Congestion Mitigationcited by 1
  4. 2024Lessons in Cooperation: A Qualitative Analysis of Driver Sentiments towards Real-Time Advisory Systems from a Driving Simulator User Studycited by 0