# Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles

arXiv: https://arxiv.org/abs/2607.08373 · HTML: https://mlsanity.com/paper/2607.08373/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 9 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [A Survey of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2312.14925/) (2023)
- [From Concept Drift to Model Degradation: An Overview on Performance-Aware Drift Detectors](https://mlsanity.com/paper/2203.11070/) (2022)
- [How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms?](https://mlsanity.com/paper/1607.01152/) (2016)
- [Factorized Q-Learning for Large-Scale Multi-Agent Systems](https://mlsanity.com/paper/1809.03738/) (2018)
- [SALAD: Self-Adaptive Lightweight Anomaly Detection for Real-time Recurrent Time Series](https://mlsanity.com/paper/2104.09968/) (2021)
- [Time Series Anomaly Detection via Reinforcement Learning-Based Model Selection](https://mlsanity.com/paper/2205.09884/) (2022)
- [Simulating Cloud Environments of Connected Vehicles for Anomaly Detection](https://mlsanity.com/paper/2404.11740/) (2024)
- [A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Series](https://mlsanity.com/paper/2405.03234/) (2024)
- [SDVDiag: A Modular Platform for the Diagnosis of Connected Vehicle Functions](https://mlsanity.com/paper/2507.19403/) (2025)
- [Directives for Function Offloading in 5G Networks Based on a Performance Characteristics Analysis](https://mlsanity.com/paper/2508.03287/) (2025)
- [AIDPS:Adaptive Intrusion Detection and Prevention System for Underwater Acoustic Sensor Networks](https://mlsanity.com/paper/2309.07730/) (2023)

## Abstract (arXiv)

Connected vehicles are autonomous cyber-physical systems whose behavior must be continuously monitored during operation to detect deviations from normal operation before they propagate into failures. Such evaluation is challenging because the systems themselves evolve: over-the-air updates, configuration changes, and shifting workloads alter the definition of normal behavior, causing static diagnostic methods to degrade silently over time. Existing approaches typically address either automated model adaptation or operator integration in isolation, rather than as a single coordinated supervisory loop. This paper presents an online anomaly detection framework for autonomous CPS that integrates three coordinated mechanisms. A factorized deep Q-network with self-attention selects the most suitable detector from a candidate pool for each monitored service, exploiting inter-service dependencies in the microservice topology. An ensemble of three statistical drift detectors monitors the input distribution and raises an alarm only when all three concur, prioritizing precision over recall. A human-in-the-loop retraining mechanism, built around a pending transition buffer and a 60/40 prioritized replay strategy, allows the operator to incorporate expert knowledge while preserving the system's learned response to prior data distributions. The framework is evaluated on a connected-vehicle testbed running an automated valet parking application across seven backend microservices. The attention-augmented agent achieves an F1 score of 0.69, compared to at most 0.11 for any single detector applied uniformly. Following a real software update that induces measurable concept drift, F1 drops to 0.52; after operator-triggered retraining, performance recovers to 0.65 on the new distribution while remaining at 0.69 on the prior one, demonstrating sustained adaptation without catastrophic forgetting.