# A Dataset Fusion Algorithm for Generalised Anomaly Detection in Homogeneous Periodic Time Series Datasets

arXiv: https://arxiv.org/abs/2305.08197 · HTML: https://mlsanity.com/paper/2305.08197/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 51 references that are in the map.
- First posted on arXiv 14 May 2023.

## Builds on (most cited first, 19 of 19)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Simple multi-dataset detection](https://mlsanity.com/paper/2102.13086/) (2021)
- [Neural Pruning via Growing Regularization](https://mlsanity.com/paper/2012.09243/) (2020)
- [Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data](https://mlsanity.com/paper/2108.09020/) (2021)
- [Domain Adaptation for Time Series Forecasting via Attention Sharing](https://mlsanity.com/paper/2102.06828/) (2021)
- [A Modern Take on the Bias-Variance Tradeoff in Neural Networks](https://mlsanity.com/paper/1810.08591/) (2018)
- [Online Adaptation to Label Distribution Shift](https://mlsanity.com/paper/2107.04520/) (2021)
- [How does Weight Correlation Affect the Generalisation Ability of Deep Neural Networks](https://mlsanity.com/paper/2010.05983/) (2020)
- [A Probabilistic Approach to Neural Network Pruning](https://mlsanity.com/paper/2105.10065/) (2021)
- [Cross-dataset Training for Class Increasing Object Detection](https://mlsanity.com/paper/2001.04621/) (2020)
- [Autoencoder-based Condition Monitoring and Anomaly Detection Method for Rotating Machines](https://mlsanity.com/paper/2101.11539/) (2021)
- [Towards an Analytical Definition of Sufficient Data](https://mlsanity.com/paper/2202.03238/) (2022)
- [Class Density and Dataset Quality in High-Dimensional, Unstructured Data](https://mlsanity.com/paper/2202.03856/) (2022)

## Abstract (arXiv)

The generalisation of Neural Networks (NN) to multiple datasets is often overlooked in literature due to NNs typically being optimised for specific data sources. This becomes especially challenging in time-series-based multi-dataset models due to difficulties in fusing sequential data from different sensors and collection specifications. In a commercial environment, however, generalisation can effectively utilise available data and computational power, which is essential in the context of Green AI, the sustainable development of AI models. This paper introduces "Dataset Fusion," a novel dataset composition algorithm for fusing periodic signals from multiple homogeneous datasets into a single dataset while retaining unique features for generalised anomaly detection. The proposed approach, tested on a case study of 3-phase current data from 2 different homogeneous Induction Motor (IM) fault datasets using an unsupervised LSTMCaps NN, significantly outperforms conventional training approaches with an Average F1 score of 0.879 and effectively generalises across all datasets. The proposed approach was also tested with varying percentages of the training data, in line with the principles of Green AI. Results show that using only 6.25% of the training data, translating to a 93.7% reduction in computational power, results in a mere 4.04% decrease in performance, demonstrating the advantages of the proposed approach in terms of both performance and computational efficiency. Moreover, the algorithm's effectiveness under non-ideal conditions highlights its potential for practical use in real-world applications.