mlsanityData to 7 Oct 2026

Sequential Anomaly Detection using Inverse Reinforcement Learning

arXiv 2004.10398 ↗First posted 22 Apr 2020cs.LG stat.MLcited by 19

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

One of the most interesting application scenarios in anomaly detection is when sequential data are targeted. For example, in a safety-critical environment, it is crucial to have an automatic detection system to screen the streaming data gathered by monitoring sensors and to report abnormal observations if detected in real-time. Oftentimes, stakes are much higher when these potential anomalies are intentional or goal-oriented. We propose an end-to-end framework for sequential anomaly detection using inverse reinforcement learning (IRL), whose objective is to determine the decision-making agent's underlying function which triggers his/her behavior. The proposed method takes the sequence of actions of a target agent (and possibly other meta information) as input. The agent's normal behavior is then understood by the reward function which is inferred via IRL. We use a neural network to represent a reward function. Using a learned reward function, we evaluate whether a new observation from the target agent follows a normal pattern. In order to construct a reliable anomaly detection method and take into consideration the confidence of the predicted anomaly score, we adopt a Bayesian approach for IRL. The empirical study on publicly available real-world data shows that our proposed method is effective in identifying anomalies.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 61 references that are in the map. First posted on arXiv 22 Apr 2020.

Builds on

15 of 61 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2015Weight Uncertainty in Neural Networkscited by 554
  5. 2016Deep Exploration via Bootstrapped DQNcited by 518
  6. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  8. 2016LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detectioncited by 213
  9. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  10. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  11. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  12. 2016Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenescited by 50
3 more
  1. 2019Crowd Counting with Decomposed Uncertaintycited by 20
  2. 2016Recurrent Neural Radio Anomaly Detectioncited by 1
  3. 2018Adaptive Pattern Matching with Reinforcement Learning for Dynamic Graphscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 2; 2023: 2; 2024: 2; 2025: 5; 2026: 3.

19 papers.

Sort:

By area:

  1. 2022FreeKD: Free-direction Knowledge Distillation for Graph Neural Networkscited by 22
  2. 2021RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learningcited by 13
  3. 2021A Survey on Deep Reinforcement Learning for Data Processing and Analyticscited by 10
  4. 2022Online Anomalous Subtrajectory Detection on Road Networks with Deep Reinforcement Learningcited by 7
  5. 2021Application of Deep Reinforcement Learning to Payment Fraudcited by 3
  6. 2024OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequencescited by 3
  7. 2025Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlationscited by 2
  8. 2021CSCAD: Correlation Structure-based Collective Anomaly Detection in Complex Systemcited by 1
  9. 2021A Survey on Proactive Customer Care: Enabling Science and Steps to Realize itcited by 0
  10. 2023Deep Reinforcement Learning for Online Error Detection in Cyber-Physical Systemscited by 0
  11. 2023Learning Representative Trajectories of Dynamical Systems via Domain-Adaptive Imitationcited by 0
  12. 2024Anomalous Change Point Detection Using Probabilistic Predictive Codingcited by 0
7 more
  1. 2025EPR-GAIL: An EPR-Enhanced Hierarchical Imitation Learning Framework to Simulate Complex User Consumption Behaviorscited by 0
  2. 2025Anomalous Decision Discovery using Inverse Reinforcement Learningcited by 0
  3. 2025Distributed Detection of Adversarial Attacks in Multi-Agent Reinforcement Learning with Continuous Action Spacecited by 0
  4. 2025Intention-aware Hierarchical Diffusion Model for Long-term Trajectory Anomaly Detectioncited by 0
  5. 2026A Context Engineering Framework for Improving Enterprise AI Agents based on Digital-Twin MDPcited by 0
  6. 2026LogNEO: A GPT-Neo Reinforcement Learning Framework for Accurate Real-Time Log Anomaly Detectioncited by 0
  7. 2026Rarity-Gated Context Conditioning for Offline Imitation Learning-Based Maritime Anomaly Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Rewards, Not Labels: Adversarial Inverse Reinforcement Learning for Machinery Fault Detection
  2. 2026Quantifying Potential Observation Missingness in Inverse Reinforcement Learning
  3. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  4. 2024Bayesian Inverse Reinforcement Learning for Non-Markovian Rewards
  5. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  6. 2026AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learning
  7. 2022Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience
  8. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review