# Towards efficient deep autoencoders for multivariate time series anomaly detection

arXiv: https://arxiv.org/abs/2403.02429 · HTML: https://mlsanity.com/paper/2403.02429/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 18 of 18)

- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Movement Pruning: Adaptive Sparsity by Fine-Tuning](https://mlsanity.com/paper/2005.07683/) (2020)
- [Graph Neural Network-Based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2106.06947/) (2021)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [Variational Dropout Sparsifies Deep Neural Networks](https://mlsanity.com/paper/1701.05369/) (2017)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [Fixed Point Quantization of Deep Convolutional Networks](https://mlsanity.com/paper/1511.06393/) (2015)
- [An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series](https://mlsanity.com/paper/2109.11428/) (2021)
- [Stabilizing the Lottery Ticket Hypothesis](https://mlsanity.com/paper/1903.01611/) (2019)
- [One-Class Adversarial Nets for Fraud Detection](https://mlsanity.com/paper/1803.01798/) (2018)
- [Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams](https://mlsanity.com/paper/1710.00811/) (2017)
- [Ensemble neuroevolution based approach for multivariate time series anomaly detection](https://mlsanity.com/paper/2108.03585/) (2021)
- [Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks](https://mlsanity.com/paper/1605.06402/) (2016)
- [Retrain or not retrain? -- efficient pruning methods of deep CNN networks](https://mlsanity.com/paper/2002.07051/) (2020)
- [Speedup deep learning models on GPU by taking advantage of efficient unstructured pruning and bit-width reduction](https://mlsanity.com/paper/2112.15445/) (2021)
- [AD-NEV: A Scalable Multi-level Neuroevolution Framework for Multivariate Anomaly Detection](https://mlsanity.com/paper/2305.16497/) (2023)

## Abstract (arXiv)

Multivariate time series anomaly detection is a crucial problem in many industrial and research applications. Timely detection of anomalies allows, for instance, to prevent defects in manufacturing processes and failures in cyberphysical systems. Deep learning methods are preferred among others for their accuracy and robustness for the analysis of complex multivariate data. However, a key aspect is being able to extract predictions in a timely manner, to accommodate real-time requirements in different applications. In the case of deep learning models, model reduction is extremely important to achieve optimal results in real-time systems with limited time and memory constraints. In this paper, we address this issue by proposing a novel compression method for deep autoencoders that involves three key factors. First, pruning reduces the number of weights, while preventing catastrophic drops in accuracy by means of a fast search process that identifies high sparsity levels. Second, linear and non-linear quantization reduces model complexity by reducing the number of bits for every single weight. The combined contribution of these three aspects allow the model size to be reduced, by removing a subset of the weights (pruning), and decreasing their bit-width (quantization). As a result, the compressed model is faster and easier to adopt in highly constrained hardware environments. Experiments performed on popular multivariate anomaly detection benchmarks, show that our method is capable of achieving significant model compression ratio (between 80% and 95%) without a significant reduction in the anomaly detection performance.