mlsanityData to 7 Oct 2026

Large AI Models in Health Informatics: Applications, Challenges, and the Future

arXiv 2303.11568 ↗First posted 21 Mar 2023cs.AI cs.CYcited by 61

Not on the map. Why?

Abstract (arXiv)

Large AI models, or foundation models, are models recently emerging with massive scales both parameter-wise and data-wise, the magnitudes of which can reach beyond billions. Once pretrained, large AI models demonstrate impressive performance in various downstream tasks. A prime example is ChatGPT, whose capability has compelled people's imagination about the far-reaching influence that large AI models can have and their potential to transform different domains of our lives. In health informatics, the advent of large AI models has brought new paradigms for the design of methodologies. The scale of multi-modal data in the biomedical and health domain has been ever-expanding especially since the community embraced the era of deep learning, which provides the ground to develop, validate, and advance large AI models for breakthroughs in health-related areas. This article presents a comprehensive review of large AI models, from background to their applications. We identify seven key sectors in which large AI models are applicable and might have substantial influence, including 1) bioinformatics; 2) medical diagnosis; 3) medical imaging; 4) medical informatics; 5) medical education; 6) public health; and 7) medical robotics. We examine their challenges, followed by a critical discussion about potential future directions and pitfalls of large AI models in transforming the field of health informatics.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 191 of its 348 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

191 of 348 references.

References · 191

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  9. 2023GPT-4 Technical Reportcited by 18,132
  10. 2022Training language models to follow instructions with human feedbackcited by 15,712
  11. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  12. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
38 more
  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  13. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  14. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  15. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  16. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  17. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  18. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  19. 2014Going Deeper with Convolutionscited by 4,231
  20. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  21. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  22. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  23. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  24. 2017Deep reinforcement learning from human preferencescited by 3,846
  25. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  26. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  27. 2022A ConvNet for the 2020scited by 3,545
  28. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  29. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  30. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  31. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  32. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  33. 2015Trust Region Policy Optimizationcited by 2,503
  34. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  35. 2022Emergent Abilities of Large Language Modelscited by 2,483
  36. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  37. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  38. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222

50 most cited of 191 All 191 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 25; 2024: 28; 2025: 8.

61 papers.

Sort:

By area:

  1. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 485
  2. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  3. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  4. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  5. 2023LEGO-Prover: Neural Theorem Proving with Growing Librariescited by 108
  6. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 101
  7. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  8. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  9. 2023Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentationcited by 51
  10. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Modelscited by 48
  11. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  12. 2024BitDelta: Your Fine-Tune May Only Be Worth One Bitcited by 38
38 more
  1. 2023ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMscited by 35
  2. 2024MEDCO: Medical Education Copilots Based on A Multi-Agent Frameworkcited by 33
  3. 2024VLMPC: Vision-Language Model Predictive Control for Robotic Manipulationcited by 32
  4. 2023Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare deliverycited by 30
  5. 2024Effectiveness Assessment of Recent Large Vision-Language Modelscited by 29
  6. 2023Lyra: Orchestrating Dual Correction in Automated Theorem Provingcited by 24
  7. 2023BioBridge: Bridging Biomedical Foundation Models via Knowledge Graphscited by 24
  8. 2023VisionFM: a Multi-Modal Multi-Task Vision Foundation Model for Generalist Ophthalmic Artificial Intelligencecited by 24
  9. 2023Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Modelcited by 24
  10. 2023Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directionscited by 14
  11. 2023PULSAR at MEDIQA-Sum 2023: Large Language Models Augmented by Synthetic Dialogue Convert Patient Dialogues to Medical Recordscited by 11
  12. 2023Dietary Assessment with Multimodal ChatGPT: A Systematic Analysiscited by 10
  13. 2024Multimodal Data Integration for Precision Oncology: Challenges and Future Directionscited by 10
  14. 2024Almanac Copilot: Towards Autonomous Electronic Health Record Navigationcited by 9
  15. 2024PhenoFlow: A Human-LLM Driven Visual Analytics System for Exploring Large and Complex Stroke Datasetscited by 9
  16. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Reviewcited by 8
  17. 2024LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasoundcited by 8
  18. 2025Emerging Cyber Attack Risks of Medical AI Agentscited by 8
  19. 2024Generalist embedding models are better at short-context clinical semantic search than specialized embedding modelscited by 7
  20. 2025SynapseRoute: An Auto-Route Switching Framework on Dual-State Large Language Modelcited by 7
  21. 2023AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentationcited by 6
  22. 2024On the Workflows and Smells of Leaderboard Operations (LBOps): An Exploratory Study of Foundation Model Leaderboardscited by 6
  23. 2024AIGC for Industrial Time Series: From Deep Generative Models to Large Generative Modelscited by 6
  24. 2023Humans are Still Better than ChatGPT: Case of the IEEEXtreme Competitioncited by 5
  25. 2023CauDR: A Causality-inspired Domain Generalization Framework for Fundus-based Diabetic Retinopathy Gradingcited by 5
  26. 2024Retrieval Augmented Thought Process for Private Data Handling in Healthcarecited by 5
  27. 2024CathAction: A Benchmark for Endovascular Intervention Understandingcited by 5
  28. 2025Science Is Falling Behind the Frontier: Foundation Model Adoption Across Half a Million Paperscited by 5
  29. 2023Utilizing ChatGPT to Enhance Clinical Trial Enrollmentcited by 4
  30. 2023A Survey on Image-text Multimodal Modelscited by 4
  31. 2023Aria-NeRF: Multimodal Egocentric View Synthesiscited by 4
  32. 2023GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathycited by 4
  33. 2024Bioinformatics and Biomedical Informatics with ChatGPT: Year One Reviewcited by 4
  34. 2024DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptationcited by 4
  35. 2024High-Throughput Phenotyping of Clinical Text Using Large Language Modelscited by 3
  36. 2025SingLEM: Single-Channel Large EEG Modelcited by 3
  37. 2023AI-SAM: Automatic and Interactive Segment Anything Modelcited by 2
  38. 2024Leveraging Large Language Models for Patient Engagement: The Power of Conversational AI in Digital Healthcited by 2

50 most cited of 61 All 61 (JSON)