mlsanityData to 7 Oct 2026

Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems

arXiv 2210.15432 ↗First posted 27 Oct 2022cs.LG cs.SEcited by 18

Not on the map. Why?

Abstract (arXiv)

Deep Neural Networks (DNNs) have been widely used to perform real-world tasks in cyber-physical systems such as Autonomous Driving Systems (ADS). Ensuring the correct behavior of such DNN-Enabled Systems (DES) is a crucial topic. Online testing is one of the promising modes for testing such systems with their application environments (simulated or real) in a closed loop taking into account the continuous interaction between the systems and their environments. However, the environmental variables (e.g., lighting conditions) that might change during the systems' operation in the real world, causing the DES to violate requirements (safety, functional), are often kept constant during the execution of an online test scenario due to the two major challenges: (1) the space of all possible scenarios to explore would become even larger if they changed and (2) there are typically many requirements to test simultaneously. In this paper, we present MORLOT (Many-Objective Reinforcement Learning for Online Testing), a novel online testing approach to address these challenges by combining Reinforcement Learning (RL) and many-objective search. MORLOT leverages RL to incrementally generate sequences of environmental changes while relying on many-objective search to determine the changes so that they are more likely to achieve any of the uncovered objectives. We empirically evaluate MORLOT using CARLA, a high-fidelity simulator widely used for autonomous driving research, integrated with Transfuser, a DNN-enabled ADS for end-to-end driving. The evaluation results show that MORLOT is significantly more effective and efficient than alternatives with a large effect size. In other words, MORLOT is a good option to test DES with dynamically changing environments while accounting for multiple safety requirements.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 12 of its 43 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

12 of 43 references.

References · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  5. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  6. 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 213
  7. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 166
  8. 2019Adaptive Stress Testing for Autonomous Vehiclescited by 70
  9. 2020Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testingcited by 40
  10. 2019Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validationcited by 33
  11. 2021Can Offline Testing of Deep Neural Networks Replace Their Online Testing?cited by 24
  12. 2021Automatic Web Testing using Curiosity-Driven Reinforcement Learningcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 3; 2025: 5; 2026: 4.

18 papers.

Sort:

By area:

  1. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
  2. 2023Boundary State Generation for Testing and Improvement of Autonomous Driving Systemscited by 6
  3. 2023EpiTESTER: Testing Autonomous Vehicles with Epigenetic Algorithm and Attention Mechanismcited by 6
  4. 2023Testing learning-enabled cyber-physical systems with Large-Language Models: A Formal Approachcited by 5
  5. 2023Evaluating the Impact of Flaky Simulators on Testing Autonomous Driving Systemscited by 4
  6. 2024Decictor: Towards Evaluating the Robustness of Decision-Making in Autonomous Driving Systemscited by 4
  7. 2023DeepQTest: Testing Autonomous Driving Systems with Reinforcement Learning and Real-world Weather Datacited by 3
  8. 2024Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Studycited by 2
  9. 2025ADReFT: Adaptive Decision Repair for Safe Autonomous Driving via Reinforcement Fine-Tuningcited by 1
  10. 2025Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directionscited by 1
  11. 2024Natural Adversaries: Fuzzing Autonomous Vehicles with Realistic Roadside Object Placementscited by 0
  12. 2025MARL-OT: Multi-Agent Reinforcement Learning Guided Online Fuzzing to Detect Safety Violation in Autonomous Driving Systemscited by 0
6 more
  1. 2025GAN-enhanced Simulation-driven DNN Testing in Absence of Ground Truthcited by 0
  2. 2025STCLocker: Deadlock Avoidance Testing for Autonomous Driving Systemscited by 0
  3. 2026Where Did It Go Wrong? Capability-Oriented Failure Attribution for Vision-and-Language Navigation Agentscited by 0
  4. 2026Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policiescited by 0
  5. 2026Evaluating Fuzz Testing for Reinforcement Learning Agentscited by 0
  6. 2026Learning from the Test: Self-Referential Differential Testing for Deep RL Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement Learning for Testing Interdependent Requirements in Autonomous Vehicles: An Empirical Study
  2. 2021Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environment
  3. 2021Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies
  4. 2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RL
  5. 2026DreamTest: World-Model Surrogates for Search-Based Testing of Deep Reinforcement Learning Agents
  6. 2025Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach
  7. 2020Deep Reinforcement Learning for Autonomous Driving: A Survey
  8. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Driving