# Fast and scalable neuroevolution deep learning architecture search for multivariate anomaly detection

arXiv: https://arxiv.org/abs/2112.05640 · HTML: https://mlsanity.com/paper/2112.05640/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 22 of 22)

- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Graph Neural Network-Based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2106.06947/) (2021)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [A review on outlier/anomaly detection in time series data](https://mlsanity.com/paper/2002.04236/) (2020)
- [A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data](https://mlsanity.com/paper/1811.08055/) (2018)
- [MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks](https://mlsanity.com/paper/1901.04997/) (2019)
- [Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection](https://mlsanity.com/paper/1802.09089/) (2018)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [Evolving Deep Neural Networks](https://mlsanity.com/paper/1703.00548/) (2017)
- [An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series](https://mlsanity.com/paper/2109.11428/) (2021)
- [Evolving Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1710.10741/) (2017)
- [One-Class Adversarial Nets for Fraud Detection](https://mlsanity.com/paper/1803.01798/) (2018)
- [Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges](https://mlsanity.com/paper/2006.05415/) (2020)
- [Ensemble neuroevolution based approach for multivariate time series anomaly detection](https://mlsanity.com/paper/2108.03585/) (2021)
- [Spectrum-Diverse Neuroevolution with Unified Neural Models](https://mlsanity.com/paper/1902.06703/) (2019)
- [MONCAE: Multi-Objective Neuroevolution of Convolutional Autoencoders](https://mlsanity.com/paper/2106.11914/) (2021)
- [Anomaly Detection using Deep Autoencoders for in-situ Wastewater Systems Monitoring Data](https://mlsanity.com/paper/2002.03843/) (2020)

## Built on by (most cited first, 2 of 2)

- [Denoising Diffusion Step-aware Models](https://mlsanity.com/paper/2310.03337/) (2023)
- [Satellite Anomaly Detection Using Variance Based Genetic Ensemble of Neural Networks](https://mlsanity.com/paper/2302.05525/) (2023)

## Abstract (arXiv)

Neuroevolution is one of the methodologies that can be used for learning optimal architecture during training. It uses evolutionary algorithms to generate the topology of artificial neural networks and its parameters. The main benefits are that it is scalable and can be fully or partially non gradient method. In this work, a modified neuroevolution technique is presented which incorporates multi-level optimisation. The presented approach adapts evolution strategies for evolving an ensemble model based on the bagging technique, using genetic operators for optimising single anomaly detection models, reducing the training dataset to speedup the search process and perform non-gradient fine tuning. Multivariate anomaly detection as an unsupervised learning task is the case study upon which the presented approach is tested. Single model optimisation is based on mutation and crossover operators and is focused on finding optimal window sizes, the number of layers, layer depths, hyperparameters etc. to boost the anomaly detection scores of new and already known models. The proposed framework and its protocol shows that it is possible to find architecture within a reasonable time frame which can boost all well known multivariate anomaly detection deep learning architectures. The work concentrates on improvements to the multi-level neuroevolution approach for anomaly detection. The main modifications are in the methods of mixing groups and single model evolution, non-gradient fine tuning and a voting mechanism. The presented framework can be used as an efficient learning network architecture method for any different unsupervised task where autoencoder architectures can be used. The tests were run on SWAT, WADI, MSL and SMAP datasets and the presented approach evolved the architectures that achieved the best scores among other deep learning models.