# Stacked Residuals of Dynamic Layers for Time Series Anomaly Detection

arXiv: https://arxiv.org/abs/2202.12457 · HTML: https://mlsanity.com/paper/2202.12457/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Classification-Based Anomaly Detection for General Data](https://mlsanity.com/paper/2005.02359/) (2020)
- [A review on outlier/anomaly detection in time series data](https://mlsanity.com/paper/2002.04236/) (2020)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting](https://mlsanity.com/paper/2010.04456/) (2020)
- [Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark](https://mlsanity.com/paper/1510.03336/) (2015)
- [Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art](https://mlsanity.com/paper/2004.00433/) (2020)
- [TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks](https://mlsanity.com/paper/2009.07769/) (2020)
- [Backprop KF: Learning Discriminative Deterministic State Estimators](https://mlsanity.com/paper/1605.07148/) (2016)
- [TAnoGAN: Time Series Anomaly Detection with Generative Adversarial Networks](https://mlsanity.com/paper/2008.09567/) (2020)
- [An Introductory Study on Time Series Modeling and Forecasting](https://mlsanity.com/paper/1302.6613/) (2013)
- [Dynamical Systems as Temporal Feature Spaces](https://mlsanity.com/paper/1907.06382/) (2019)
- [A novel Deep Neural Network architecture for non-linear system identification](https://mlsanity.com/paper/2106.03078/) (2021)
- [Less is More: Building Selective Anomaly Ensembles](https://mlsanity.com/paper/1501.01924/) (2015)

## Built on by (most cited first, 4 of 4)

- [B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory](https://mlsanity.com/paper/2407.06324/) (2024)
- [Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression](https://mlsanity.com/paper/2511.21016/) (2025)
- [STree: Speculative Tree Decoding for Hybrid State-Space Models](https://mlsanity.com/paper/2505.14969/) (2025)
- [Priming: Hybrid State Space Models From Pre-trained Transformers](https://mlsanity.com/paper/2605.08301/) (2026)

## Abstract (arXiv)

We present an end-to-end differentiable neural network architecture to perform anomaly detection in multivariate time series by incorporating a Sequential Probability Ratio Test on the prediction residual. The architecture is a cascade of dynamical systems designed to separate linearly predictable components of the signal such as trends and seasonality, from the non-linear ones. The former are modeled by local Linear Dynamic Layers, and their residual is fed to a generic Temporal Convolutional Network that also aggregates global statistics from different time series as context for the local predictions of each one. The last layer implements the anomaly detector, which exploits the temporal structure of the prediction residuals to detect both isolated point anomalies and set-point changes. It is based on a novel application of the classic CUMSUM algorithm, adapted through the use of a variational approximation of f-divergences. The model automatically adapts to the time scales of the observed signals. It approximates a SARIMA model at the get-go, and auto-tunes to the statistics of the signal and its covariates, without the need for supervision, as more data is observed. The resulting system, which we call STRIC, outperforms both state-of-the-art robust statistical methods and deep neural network architectures on multiple anomaly detection benchmarks.