Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlations
Not on the map. Why?
Abstract (arXiv)
Nowadays, as AI-driven manufacturing becomes increasingly popular, the volume of data streams requiring real-time monitoring continues to grow. However, due to limited resources, it is impractical to place sensors at every location to detect unexpected shifts. Therefore, it is necessary to develop an optimal sensor placement strategy that enables partial observability of the system while detecting anomalies as quickly as possible. Numerous approaches have been proposed to address this challenge; however, most existing methods consider only variable correlations and neglect a crucial factor: Causality. Moreover, although a few techniques incorporate causal analysis, they rely on interventions-artificially creating anomalies-to identify causal effects, which is impractical and might lead to catastrophic losses. In this paper, we introduce a causality-informed deep Q-network (Causal DQ) approach for partially observable sensor placement in anomaly detection. By integrating causal information at each stage of Q-network training, our method achieves faster convergence and tighter theoretical error bounds. Furthermore, the trained causal-informed Q-network significantly reduces the detection time for anomalies under various settings, demonstrating its effectiveness for sensor placement in large-scale, real-world data streams. Beyond the current implementation, our technique's fundamental insights can be applied to various reinforcement learning problems, opening up new possibilities for real-world causality-informed machine learning methods in engineering applications.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 37 of its 75 references that are in the map. First posted on arXiv 13 Jul 2025.
Builds on
37 of 75 references.
References · 37
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 752
- 2019A Theory of Regularized Markov Decision Processescited by 286
- 2019A Theoretical Analysis of Deep Q-Learningcited by 215
- 2019Causal Discovery with Reinforcement Learningcited by 145
- 2022Causal Machine Learning: A Survey and Open Problemscited by 110
- 2022When Is Partially Observable Reinforcement Learning Not Scary?cited by 100
- 2021A survey of Bayesian Network structure learningcited by 87
- 2022Partially Observable Markov Decision Processes in Robotics: A Surveycited by 86
- 2024Causal machine learning for predicting treatment outcomescited by 74
- 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
- 2021Ordering-Based Causal Discovery with Reinforcement Learningcited by 55
25 more
- 2021Partially Observable Markov Decision Processes (POMDPs) and Roboticscited by 49
- 2022Large-Scale Differentiable Causal Discovery of Factor Graphscited by 47
- 2023A Survey on Causal Reinforcement Learningcited by 45
- 2021Causal Reinforcement Learning using Observational and Interventional Datacited by 39
- 2021Instrumental Variable Value Iteration for Causal Offline Reinforcement Learningcited by 35
- 2020Meta-AAD: Active Anomaly Detection with Deep Reinforcement Learningcited by 34
- 2020Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Frameworkcited by 29
- 2022Sample-Efficient Reinforcement Learning of Partially Observable Markov Gamescited by 27
- 2022Evaluation Methods and Measures for Causal Learning Algorithmscited by 23
- 2023Causal Reinforcement Learning: A Surveycited by 23
- 2020Sequential Anomaly Detection using Inverse Reinforcement Learningcited by 20
- 2021A Discrete-Time Switching System Analysis of Q-learningcited by 16
- 2021Causal Multi-Agent Reinforcement Learning: Review and Open Problemscited by 16
- 2024ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularizationcited by 16
- 2019Reinforcement Learning with Dynamic Boltzmann Softmax Updatescited by 14
- 2023Computably Continuous Reinforcement-Learning Objectives are PAC-learnablecited by 13
- 2020Partially Observable Minimum-Age Scheduling: The Greedy Policycited by 5
- 2020Bandit Change-Point Detection for Real-Time Monitoring High-Dimensional Data Under Sampling Controlcited by 4
- 2024Causal Change Point Detection and Localizationcited by 4
- 2024Boosting Soft Q-Learning by Boundingcited by 4
- 2025Causal Information Prioritization for Efficient Reinforcement Learningcited by 4
- 2022Causality, Causal Discovery, and Causal Inference in Structural Engineeringcited by 3
- 2024Finite-Time Error Analysis of Soft Q-Learning: Switching System Approachcited by 3
- 2024Causal Discovery-Driven Change Point Detection in Time Seriescited by 2
- 2025Explainable Federated Bayesian Causal Inference and Its Application in Advanced Manufacturingcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Semi-supervised Anomaly Detection via Adaptive Reinforcement Learning-Enabled Method with Causal Inference for Sensor Signals
- 2026CausalPulse: An Industrial-Grade Neurosymbolic Multi-Agent Copilot for Causal Diagnostics in Smart Manufacturing
- 2025Diffuse to Detect: A Generalizable Framework for Anomaly Detection with Diffusion Models Applications to UAVs and Beyond
- 2024Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation
- 2025Dynamic Reward Scaling for Multivariate Time Series Anomaly Detection: A VAE-Enhanced Reinforcement Learning Approach
- 2024COKE: Causal Discovery with Chronological Order and Expert Knowledge in High Proportion of Missing Manufacturing Data
- 2026ASTRO: Adaptive Spatio-Temporal Reinforcement Optimization for GNN Powered Anomly Detection in Cyber Physical Systems
- 2024Efficient Discovery of Actual Causality using Abstraction-Refinement