# Dual Aspect Self-Attention based on Transformer for Remaining Useful Life Prediction

arXiv: https://arxiv.org/abs/2106.15842 · HTML: https://mlsanity.com/paper/2106.15842/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 43 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case](https://mlsanity.com/paper/2001.08317/) (2020)
- [Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder](https://mlsanity.com/paper/1608.06154/) (2016)

## Built on by (most cited first, 16 of 16)

- [Interpretable Prognostics with Concept Bottleneck Models](https://mlsanity.com/paper/2405.17575/) (2024)
- [Survey on Foundation Models for Prognostics and Health Management in Industrial Cyber-Physical Systems](https://mlsanity.com/paper/2312.06261/) (2023)
- [LAMA-Net: Unsupervised Domain Adaptation via Latent Alignment and Manifold Learning for RUL Prediction](https://mlsanity.com/paper/2208.08388/) (2022)
- [Multi-Dimensional Self Attention based Approach for Remaining Useful Life Estimation](https://mlsanity.com/paper/2212.05772/) (2022)
- [TFBEST: Dual-Aspect Transformer with Learnable Positional Encoding for Failure Prediction](https://mlsanity.com/paper/2309.02641/) (2023)
- [Spatio-temporal Attention-based Hidden Physics-informed Neural Network for Remaining Useful Life Prediction](https://mlsanity.com/paper/2405.12377/) (2024)
- [Sparse Low-Ranked Self-Attention Transformer for Remaining Useful Lifetime Prediction of Optical Fiber Amplifiers](https://mlsanity.com/paper/2409.14378/) (2024)
- [Domain Adaptation-based Edge Computing for Cross-Conditions Fault Diagnosis](https://mlsanity.com/paper/2411.10340/) (2024)
- [A Transformer-Based Approach for Diagnosing Fault Cases in Optical Fiber Amplifiers](https://mlsanity.com/paper/2505.06245/) (2025)
- [PerFormer: A Permutation Based Vision Transformer for Remaining Useful Life Prediction](https://mlsanity.com/paper/2506.00259/) (2025)
- [Transformer-Based Prognostics: Enhancing Network Availability by Improved Monitoring of Optical Fiber Amplifiers](https://mlsanity.com/paper/2603.28081/) (2026)
- [Generalization bounds and sample complexity for remaining useful life prediction from complete degradation trajectories](https://mlsanity.com/paper/2607.23454/) (2026)
- [Collaborative System Failure Prognostics via Federated Longitudinal-Survival Modeling](https://mlsanity.com/paper/2607.26038/) (2026)
- [PhysicsBench: A Unified Leaderboard for Generative and Predictive Models in Engineering Design and Simulation](https://mlsanity.com/paper/2608.24056/) (2026)
- [Multi-Term Fourier Graph Neural Network with Sample Relationship Learning for Enhanced Remaining Useful Life Prediction](https://mlsanity.com/paper/2609.25179/) (2026)
- [Calibrating Prediction Timeliness Through Multi-Objective Hyperparameter Optimization for Remaining Useful Life Prediction](https://mlsanity.com/paper/2610.01530/) (2026)

## Abstract (arXiv)

Remaining useful life prediction (RUL) is one of the key technologies of condition-based maintenance, which is important to maintain the reliability and safety of industrial equipments. Massive industrial measurement data has effectively improved the performance of the data-driven based RUL prediction method. While deep learning has achieved great success in RUL prediction, existing methods have difficulties in processing long sequences and extracting information from the sensor and time step aspects. In this paper, we propose Dual Aspect Self-attention based on Transformer (DAST), a novel deep RUL prediction method, which is an encoder-decoder structure purely based on self-attention without any RNN/CNN module. DAST consists of two encoders, which work in parallel to simultaneously extract features of different sensors and time steps. Solely based on self-attention, the DAST encoders are more effective in processing long data sequences, and are capable of adaptively learning to focus on more important parts of input. Moreover, the parallel feature extraction design avoids mutual influence of information from two aspects. Experiments on two widely used turbofan engines datasets show that our method significantly outperforms the state-of-the-art RUL prediction methods.