mlsanityData to 14 Oct 2026

ChatGPT-Like Large-Scale Foundation Models for Prognostics and Health Management: A Survey and Roadmaps

arXiv 2305.06472 ↗First posted 10 May 2023cs.LG cs.AI cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Prognostics and health management (PHM) technology plays a critical role in industrial production and equipment maintenance by identifying and predicting possible equipment failures and damages, thereby allowing necessary maintenance measures to be taken to enhance equipment service life and reliability while reducing production costs and downtime. In recent years, PHM technology based on artificial intelligence (AI) has made remarkable achievements in the context of the industrial IoT and big data, and it is widely used in various industries, such as railway, energy, and aviation, for condition monitoring, fault prediction, and health management. The emergence of large-scale foundation models (LSF-Models) such as ChatGPT and DALLE-E marks the entry of AI into a new era of AI-2.0 from AI-1.0, where deep models have rapidly evolved from a research paradigm of single-modal, single-task, and limited-data to a multi-modal, multi-task, massive data, and super-large model paradigm. ChatGPT represents a landmark achievement in this research paradigm, offering hope for general artificial intelligence due to its highly intelligent natural language understanding ability. However, the PHM field lacks a consensus on how to respond to this significant change in the AI field, and a systematic review and roadmap is required to elucidate future development directions. To fill this gap, this paper systematically expounds on the key components and latest developments of LSF-Models. Then, we systematically answered how to build the LSF-Model applicable to PHM tasks and outlined the challenges and future development roadmaps for this research paradigm.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 137 of its 306 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

137 of 306 references.

References · 137

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  7. 2023GPT-4 Technical Reportcited by 18,396
  8. 2022Training language models to follow instructions with human feedbackcited by 15,971
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  11. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  12. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
38 more
  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2023Segment Anythingcited by 8,234
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2016Layer Normalizationcited by 5,669
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  12. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  13. 2020Learning to summarize from human feedbackcited by 2,588
  14. 2023A Survey of Large Language Modelscited by 2,576
  15. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  16. 2020GLU Variants Improve Transformercited by 1,804
  17. 2020Reformer: The Efficient Transformercited by 1,668
  18. 2020Rethinking Attention with Performerscited by 1,601
  19. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  20. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  21. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  22. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  23. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  24. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  25. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
  26. 2021Video Swin Transformercited by 1,091
  27. 2021Transformers in Vision: A Surveycited by 1,013
  28. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  29. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  30. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  31. 2020A Survey on Visual Transformercited by 654
  32. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  33. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  34. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  35. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  36. 2022Transformers in Time Series: A Surveycited by 497
  37. 2021A Survey of Transformerscited by 401
  38. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377

50 most cited of 137 All 137 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 1; 2026: 1.

8 papers.

  1. 2023Large Scale Foundation Models for Intelligent Manufacturing Applications: A Surveycited by 6
  2. 2024BearLLM: A Prior Knowledge-Enhanced Bearing Health Management Framework with Unified Vibration Signal Representationcited by 4
  3. 2024RmGPT: A Foundation Model with Generative Pre-trained Transformer for Fault Diagnosis and Prognosis in Rotating Machinerycited by 3
  4. 2023Machine Learning Approaches for Diagnostics and Prognostics of Industrial Systems Using Open Source Data from PHM Data Challenges: A Reviewcited by 2
  5. 2023Empowering ChatGPT-Like Large-Scale Language Models with Local Knowledge Base for Industrial Prognostics and Health Managementcited by 1
  6. 2024VSLLaVA: a pipeline of large multimodal foundation model for industrial vibration signal analysiscited by 0
  7. 2025Engineering Artificial Intelligence: Framework, Challenges, and Future Directioncited by 0
  8. 2026Generalization bounds and sample complexity for remaining useful life prediction from complete degradation trajectoriescited by 0