mlsanityData to 7 Oct 2026

Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation

arXiv 2210.03209 ↗First posted 6 Oct 2022cs.ROcited by 12

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Powered by deep representation learning, reinforcement learning (RL) provides an end-to-end learning framework capable of solving self-driving (SD) tasks without manual designs. However, time-varying nonstationary environments cause proficient but specialized RL policies to fail at execution time. For example, an RL-based SD policy trained under sunny days does not generalize well to rainy weather. Even though meta learning enables the RL agent to adapt to new tasks/environments, its offline operation fails to equip the agent with online adaptation ability when facing nonstationary environments. This work proposes an online meta reinforcement learning algorithm based on the conjectural online lookahead adaptation (COLA). COLA determines the online adaptation at every step by maximizing the agent's conjecture of the future performance in a lookahead horizon. Experimental results demonstrate that under dynamically changing weather and lighting conditions, the COLA-based self-adaptive driving outperforms the baseline policies in terms of online adaptability. A demo video, source code, and appendixes are available at tt https://github.com/Panshark/COLA

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 25 of its 41 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

25 of 41 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  6. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  7. 2017Constrained Policy Optimizationcited by 928
  8. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  9. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
  10. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  11. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  12. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 172
13 more
  1. 2019Online Meta-Learningcited by 137
  2. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
  3. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  4. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  5. 2019Continuous Meta-Learning without Taskscited by 50
  6. 2018A Reinforcement Learning Based Approach for Automated Lane Change Maneuverscited by 44
  7. 2014Online Planning Algorithms for POMDPscited by 42
  8. 2020Deep Reinforcement Learning amidst Lifelong Non-Stationaritycited by 42
  9. 2021Generalized Proximal Policy Optimization with Sample Reusecited by 39
  10. 2016End-to-End Deep Reinforcement Learning for Lane Keeping Assistcited by 27
  11. 2021The Confluence of Networks, Games and Learningcited by 26
  12. 2019On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learningcited by 12
  13. 2020Blackwell Online Learning for Markov Decision Processescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 3; 2025: 4; 2026: 3.

12 papers.

  1. 2024Symbiotic Game and Foundation Models for Cyber Deception Operations in Strategic Cyber Warfarecited by 7
  2. 2024Multi-level Traffic-Responsive Tilt Camera Surveillance through Predictive Correlated Online Learningcited by 5
  3. 2025Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigationcited by 4
  4. 2026In-Context Autonomous Network Incident Response: An End-to-End Large Language Model Agent Approachcited by 4
  5. 2025From Texts to Shields: Convergence of Large Language Models and Cybersecuritycited by 3
  6. 2025In-Context Reinforcement Learning via Communicative World Modelscited by 3
  7. 2024Digital Twin-based Driver Risk-Aware Intelligent Mobility Analytics for Urban Transportation Managementcited by 2
  8. 2023Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environmentscited by 1
  9. 2025Stackelberg Coupling of Online Representation Learning and Reinforcement Learningcited by 1
  10. 2026Agentic Incident Response through Digital Twin-Enhanced Multiscale Planningcited by 1
  11. 2022Cognitive Level-k Meta-Learning for Safe and Pedestrian-Aware Autonomous Drivingcited by 0
  12. 2026Hierarchical Agentic Incident Response with Digital-Twin-Validated Attack Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Control in Autonomous Driving via Real-Time Recurrent RL
  2. 2024From Imitation to Exploration: End-to-end Autonomous Driving based on World Model
  3. 2021Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learning
  4. 2025Prompt-Driven Domain Adaptation for End-to-End Autonomous Driving via In-Context RL
  5. 2025Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environments
  6. 2021Learning robust driving policies without online exploration
  7. 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models
  8. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning