# Predictive Maintenance for General Aviation Using Convolutional Transformers

arXiv: https://arxiv.org/abs/2110.03757 · HTML: https://mlsanity.com/paper/2110.03757/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [Time Series Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2002.12478/) (2020)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks](https://mlsanity.com/paper/2007.15951/) (2020)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [Attend and Diagnose: Clinical Time Series Analysis using Attention Models](https://mlsanity.com/paper/1711.03905/) (2017)
- [Self-attention for raw optical Satellite Time Series Classification](https://mlsanity.com/paper/1910.10536/) (2019)
- [Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition](https://mlsanity.com/paper/1512.06747/) (2015)
- [Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMs](https://mlsanity.com/paper/1603.00423/) (2016)
- [Convolutional RNN: an Enhanced Model for Extracting Features from Sequential Data](https://mlsanity.com/paper/1602.05875/) (2016)
- [MaintNet: A Collaborative Open-Source Library for Predictive Maintenance Language Resources](https://mlsanity.com/paper/2005.12443/) (2020)

## Built on by (most cited first, 6 of 6)

- [FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes](https://mlsanity.com/paper/2506.03278/) (2025)
- [Robust Augmentation for Multivariate Time Series Classification](https://mlsanity.com/paper/2201.11739/) (2022)
- [A Large-Scale Annotated Multivariate Time Series Aviation Maintenance Dataset from the NGAFID](https://mlsanity.com/paper/2210.07317/) (2022)
- [A Compliance-Preserving Retrieval System for Aircraft MRO Task Search](https://mlsanity.com/paper/2511.15383/) (2025)
- [A Heterogeneous Long-Micro Scale Cascading Architecture for General Aviation Health Management](https://mlsanity.com/paper/2603.22885/) (2026)
- [Triple Spectral Fusion for Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2605.02743/) (2026)

## Abstract (arXiv)

Predictive maintenance systems have the potential to significantly reduce costs for maintaining aircraft fleets as well as provide improved safety by detecting maintenance issues before they come severe. However, the development of such systems has been limited due to a lack of publicly labeled multivariate time series (MTS) sensor data. MTS classification has advanced greatly over the past decade, but there is a lack of sufficiently challenging benchmarks for new methods. This work introduces the NGAFID Maintenance Classification (NGAFID-MC) dataset as a novel benchmark in terms of difficulty, number of samples, and sequence length. NGAFID-MC consists of over 7,500 labeled flights, representing over 11,500 hours of per second flight data recorder readings of 23 sensor parameters. Using this benchmark, we demonstrate that Recurrent Neural Network (RNN) methods are not well suited for capturing temporally distant relationships and propose a new architecture called Convolutional Multiheaded Self Attention (Conv-MHSA) that achieves greater classification performance at greater computational efficiency. We also demonstrate that image inspired augmentations of cutout, mixup, and cutmix, can be used to reduce overfitting and improve generalization in MTS classification. Our best trained models have been incorporated back into the NGAFID to allow users to potentially detect flights that require maintenance as well as provide feedback to further expand and refine the NGAFID-MC dataset.