# Improving Semi-Supervised Learning for Remaining Useful Lifetime Estimation Through Self-Supervision

arXiv: https://arxiv.org/abs/2108.08721 · HTML: https://mlsanity.com/paper/2108.08721/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [Semi-Supervised Learning with Ladder Networks](https://mlsanity.com/paper/1507.02672/) (2015)
- [Learning with Pseudo-Ensembles](https://mlsanity.com/paper/1412.4864/) (2014)
- [Wasserstein Distance based Deep Adversarial Transfer Learning for Intelligent Fault Diagnosis](https://mlsanity.com/paper/1903.06753/) (2019)
- [Semi-supervised Learning with Deep Generative Models for Asset Failure Prediction](https://mlsanity.com/paper/1709.00845/) (2017)

## Built on by (most cited first, 3 of 3)

- [Masked Self-Supervision for Remaining Useful Lifetime Prediction in Machine Tools](https://mlsanity.com/paper/2207.01219/) (2022)
- [Self-Supervised Learning for Data Scarcity in a Fatigue Damage Prognostic Problem](https://mlsanity.com/paper/2301.08441/) (2023)
- [The State of Lithium-Ion Battery Health Prognostics in the CPS Era](https://mlsanity.com/paper/2403.19816/) (2024)

## Abstract (arXiv)

RUL estimation suffers from a server data imbalance where data from machines near their end of life is rare. Additionally, the data produced by a machine can only be labeled after the machine failed. Semi-Supervised Learning (SSL) can incorporate the unlabeled data produced by machines that did not yet fail. Previous work on SSL evaluated their approaches under unrealistic conditions where the data near failure was still available. Even so, only moderate improvements were made. This paper proposes a novel SSL approach based on self-supervised pre-training. The method can outperform two competing approaches from the literature and a supervised baseline under realistic conditions on the NASA C-MAPSS dataset.