# Big Machinery Data Preprocessing Methodology for Data-Driven Models in Prognostics and Health Management

arXiv: https://arxiv.org/abs/2110.04256 · HTML: https://mlsanity.com/paper/2110.04256/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 71 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 8 of 8)

- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Time Series Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2002.12478/) (2020)
- [Deep Learning for Anomaly Detection: A Review](https://mlsanity.com/paper/2007.02500/) (2020)
- [A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective](https://mlsanity.com/paper/1811.03402/) (2018)
- [Learning the Structure of Generative Models without Labeled Data](https://mlsanity.com/paper/1703.00854/) (2017)
- [Improving the Accuracy of Global Forecasting Models using Time Series Data Augmentation](https://mlsanity.com/paper/2008.02663/) (2020)
- [Smart Anomaly Detection in Sensor Systems: A Multi-Perspective Review](https://mlsanity.com/paper/2010.14946/) (2020)
- [Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder](https://mlsanity.com/paper/1608.06154/) (2016)

## Built on by (most cited first, 3 of 3)

- [Machine Learning Approaches for Diagnostics and Prognostics of Industrial Systems Using Open Source Data from PHM Data Challenges: A Review](https://mlsanity.com/paper/2312.16810/) (2023)
- [Rethinking industrial artificial intelligence: a unified foundation framework](https://mlsanity.com/paper/2504.01797/) (2025)
- [Prognostics for Autonomous Deep-Space Habitat Health Management under Multiple Unknown Failure Modes](https://mlsanity.com/paper/2411.12159/) (2024)

## Abstract (arXiv)

Sensor monitoring networks and advances in big data analytics have guided the reliability engineering landscape to a new era of big machinery data. Low-cost sensors, along with the evolution of the internet of things and industry 4.0, have resulted in rich databases that can be analyzed through prognostics and health management (PHM) frameworks. Several da-ta-driven models (DDMs) have been proposed and applied for diagnostics and prognostics purposes in complex systems. However, many of these models are developed using simulated or experimental data sets, and there is still a knowledge gap for applications in real operating systems. Furthermore, little attention has been given to the required data preprocessing steps compared to the training processes of these DDMs. Up to date, research works do not follow a formal and consistent data preprocessing guideline for PHM applications. This paper presents a comprehensive, step-by-step pipeline for the preprocessing of monitoring data from complex systems aimed for DDMs. The importance of expert knowledge is discussed in the context of data selection and label generation. Two case studies are presented for validation, with the end goal of creating clean data sets with healthy and unhealthy labels that are then used to train machinery health state classifiers.