# Semi-supervised Anomaly Detection via Adaptive Reinforcement Learning-Enabled Method with Causal Inference for Sensor Signals

arXiv: https://arxiv.org/abs/2405.06925 · HTML: https://mlsanity.com/paper/2405.06925/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 48 references that are in the map.
- First posted on arXiv 11 May 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 14 of 14)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [Classification-Based Anomaly Detection for General Data](https://mlsanity.com/paper/2005.02359/) (2020)
- [Deep Anomaly Detection with Deviation Networks](https://mlsanity.com/paper/1911.08623/) (2019)
- [Interventional Causal Representation Learning](https://mlsanity.com/paper/2209.11924/) (2022)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Toward Supervised Anomaly Detection](https://mlsanity.com/paper/1401.6424/) (2014)
- [Deep Weakly-supervised Anomaly Detection](https://mlsanity.com/paper/1910.13601/) (2019)
- [Rethinking Assumptions in Deep Anomaly Detection](https://mlsanity.com/paper/2006.00339/) (2020)
- [Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data](https://mlsanity.com/paper/2009.06847/) (2020)
- [Supervised Anomaly Detection via Conditional Generative Adversarial Network and Ensemble Active Learning](https://mlsanity.com/paper/2104.11952/) (2021)
- [RoSAS: Deep Semi-Supervised Anomaly Detection with Contamination-Resilient Continuous Supervision](https://mlsanity.com/paper/2307.13239/) (2023)
- [Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning](https://mlsanity.com/paper/2212.01174/) (2022)

## Built on by (most cited first, 2 of 2)

- [Incremental Causal Graph Learning for Online Cyberattack Detection in Cyber-Physical Infrastructures](https://mlsanity.com/paper/2507.14387/) (2025)
- [LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2601.02511/) (2026)

## Abstract (arXiv)

Semi-supervised anomaly detection for sensor signals is critical in ensuring system reliability in smart manufacturing. However, existing methods rely heavily on data correlation, neglecting causality and leading to potential misinterpretations due to confounding factors. Moreover, while current reinforcement learning-based methods can effectively identify known and unknown anomalies with limited labeled samples, these methods still face several challenges, such as under-utilization of priori knowledge, lack of model flexibility, and deficient reward feedback during environmental interactions. To address the above problems, this paper innovatively constructs a counterfactual causal reinforcement learning model, termed Triple-Assisted Causal Reinforcement Learning Anomaly Detector (Tri-CRLAD). The model leverages causal inference to extract the intrinsic causal feature in data, enhancing the agent's utilization of prior knowledge and improving its generalization capability. In addition, Tri-CRLAD features a triple decision support mechanism, including a sampling strategy based on historical similarity, an adaptive threshold smoothing adjustment strategy, and an adaptive decision reward mechanism. These mechanisms further enhance the flexibility and generalization ability of the model, enabling it to effectively respond to various complex and dynamically changing environments. Experimental results across seven diverse sensor signal datasets demonstrate that Tri-CRLAD outperforms nine state-of-the-art baseline methods. Notably, Tri-CRLAD achieves up to a 23% improvement in anomaly detection stability with minimal known anomaly samples, highlighting its potential in semi-supervised anomaly detection scenarios. Our code is available at https://github.com/Aoudsung/Tri-CRLAD.