# Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection

arXiv: https://arxiv.org/abs/2106.12382 · HTML: https://mlsanity.com/paper/2106.12382/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 62 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [Forecasting Sequential Data using Consistent Koopman Autoencoders](https://mlsanity.com/paper/2003.02236/) (2020)
- [BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling](https://mlsanity.com/paper/1902.02102/) (2019)
- [One-Class Classification: Taxonomy of Study and Review of Techniques](https://mlsanity.com/paper/1312.0049/) (2013)
- [Deep Dynamical Modeling and Control of Unsteady Fluid Flows](https://mlsanity.com/paper/1805.07472/) (2018)
- [Learning Stochastic Recurrent Networks](https://mlsanity.com/paper/1411.7610/) (2014)
- [Z-Forcing: Training Stochastic Recurrent Networks](https://mlsanity.com/paper/1711.05411/) (2017)
- [Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives](https://mlsanity.com/paper/1810.04152/) (2018)
- [Perfect density models cannot guarantee anomaly detection](https://mlsanity.com/paper/2012.03808/) (2020)
- [Learning Independent Features with Adversarial Nets for Non-linear ICA](https://mlsanity.com/paper/1710.05050/) (2017)
- [Achieving the Dispatchability of Distribution Feeders through Prosumers Data Driven Forecasting and Model Predictive Control of Electrochemical Storage](https://mlsanity.com/paper/1602.02265/) (2016)
- [Universal Data Anomaly Detection via Inverse Generative Adversary Network](https://mlsanity.com/paper/2001.08809/) (2020)
- [Adaptive Subband Compression for Streaming of Continuous Point-on-Wave and PMU Data](https://mlsanity.com/paper/2008.10092/) (2020)

## Built on by (most cited first, 7 of 7)

- [Rank Reduction Autoencoders](https://mlsanity.com/paper/2405.13980/) (2024)
- [Non-parametric Probabilistic Time Series Forecasting via Innovations Representation](https://mlsanity.com/paper/2306.03782/) (2023)
- [Novelty Detection in Time Series via Weak Innovations Representation: A Deep Learning Approach](https://mlsanity.com/paper/2210.13358/) (2022)
- [Generative Probabilistic Time Series Forecasting and Applications in Grid Operations](https://mlsanity.com/paper/2402.13870/) (2024)
- [Grid Monitoring with Synchro-Waveform and AI Foundation Model Technologies](https://mlsanity.com/paper/2403.06942/) (2024)
- [AI Foundation Model for Time Series with Innovations Representation](https://mlsanity.com/paper/2510.01560/) (2025)
- [Learning to Test: Latent Representation Learning for Dynamic Instability Detection](https://mlsanity.com/paper/2604.10967/) (2026)

## Abstract (arXiv)

An innovations sequence of a time series is a sequence of independent and identically distributed random variables with which the original time series has a causal representation. The innovation at a time is statistically independent of the history of the time series. As such, it represents the new information contained at present but not in the past. Because of its simple probability structure, an innovations sequence is the most efficient signature of the original. Unlike the principle or independent component analysis representations, an innovations sequence preserves not only the complete statistical properties but also the temporal order of the original time series. An long-standing open problem is to find a computationally tractable way to extract an innovations sequence of non-Gaussian processes. This paper presents a deep learning approach, referred to as Innovations Autoencoder (IAE), that extracts innovations sequences using a causal convolutional neural network. An application of IAE to the one-class anomalous sequence detection problem with unknown anomaly and anomaly-free models is also presented.