# ChatGPT-Like Large-Scale Foundation Models for Prognostics and Health Management: A Survey and Roadmaps arXiv: https://arxiv.org/abs/2305.06472 ยท HTML: https://mlsanity.com/paper/2305.06472/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 137 of its 306 references that are in the map. - First posted on arXiv 10 May 2023. ## Builds on (most cited first, 30 of 137) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018) - [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019) - [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016) - [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021) - [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022) - [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019) - [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019) - [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020) - [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023) - [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020) - [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020) - [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020) - [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020) ## Built on by (most cited first, 8 of 8) - [Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey](https://mlsanity.com/paper/2312.06718/) (2023) - [BearLLM: A Prior Knowledge-Enhanced Bearing Health Management Framework with Unified Vibration Signal Representation](https://mlsanity.com/paper/2408.11281/) (2024) - [RmGPT: A Foundation Model with Generative Pre-trained Transformer for Fault Diagnosis and Prognosis in Rotating Machinery](https://mlsanity.com/paper/2409.17604/) (2024) - [Machine Learning Approaches for Diagnostics and Prognostics of Industrial Systems Using Open Source Data from PHM Data Challenges: A Review](https://mlsanity.com/paper/2312.16810/) (2023) - [Empowering ChatGPT-Like Large-Scale Language Models with Local Knowledge Base for Industrial Prognostics and Health Management](https://mlsanity.com/paper/2312.14945/) (2023) - [VSLLaVA: a pipeline of large multimodal foundation model for industrial vibration signal analysis](https://mlsanity.com/paper/2409.07482/) (2024) - [Engineering Artificial Intelligence: Framework, Challenges, and Future Direction](https://mlsanity.com/paper/2504.02269/) (2025) - [Generalization bounds and sample complexity for remaining useful life prediction from complete degradation trajectories](https://mlsanity.com/paper/2607.23454/) (2026) ## Abstract (arXiv) Prognostics and health management (PHM) technology plays a critical role in industrial production and equipment maintenance by identifying and predicting possible equipment failures and damages, thereby allowing necessary maintenance measures to be taken to enhance equipment service life and reliability while reducing production costs and downtime. In recent years, PHM technology based on artificial intelligence (AI) has made remarkable achievements in the context of the industrial IoT and big data, and it is widely used in various industries, such as railway, energy, and aviation, for condition monitoring, fault prediction, and health management. The emergence of large-scale foundation models (LSF-Models) such as ChatGPT and DALLE-E marks the entry of AI into a new era of AI-2.0 from AI-1.0, where deep models have rapidly evolved from a research paradigm of single-modal, single-task, and limited-data to a multi-modal, multi-task, massive data, and super-large model paradigm. ChatGPT represents a landmark achievement in this research paradigm, offering hope for general artificial intelligence due to its highly intelligent natural language understanding ability. However, the PHM field lacks a consensus on how to respond to this significant change in the AI field, and a systematic review and roadmap is required to elucidate future development directions. To fill this gap, this paper systematically expounds on the key components and latest developments of LSF-Models. Then, we systematically answered how to build the LSF-Model applicable to PHM tasks and outlined the challenges and future development roadmaps for this research paradigm.