mlsanityData to 14 Oct 2026

Health AI Developer Foundations

arXiv 2411.15128 ↗First posted 22 Nov 2024cs.LG cs.AI cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Robust medical Machine Learning (ML) models have the potential to revolutionize healthcare by accelerating clinical research, improving workflows and outcomes, and producing novel insights or capabilities. Developing such ML models from scratch is cost prohibitive and requires substantial compute, data, and time (e.g., expert labeling). To address these challenges, we introduce Health AI Developer Foundations (HAI-DEF), a suite of pre-trained, domain-specific foundation models, tools, and recipes to accelerate building ML for health applications. The models cover various modalities and domains, including radiology (X-rays and computed tomography), histopathology, dermatological imaging, and audio. These models provide domain specific embeddings that facilitate AI development with less labeled data, shorter training times, and reduced computational costs compared to traditional approaches. In addition, we utilize a common interface and style across these models, and prioritize usability to enable developers to integrate HAI-DEF efficiently. We present model evaluations across various tasks and conclude with a discussion of their application and evaluation, covering the importance of ensuring efficacy, fairness, and equity. Finally, while HAI-DEF and specifically the foundation models lower the barrier to entry for ML in healthcare, we emphasize the importance of validation with problem- and population-specific data for each desired usage setting. This technical report will be updated over time as more modalities and features are added.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 22 Nov 2024.

Builds on

20 of 29 references.

References · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2020Supervised Contrastive Learningcited by 2,829
  6. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  7. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  8. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  9. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  10. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
  11. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  12. 2024Advancing Multimodal Medical Capabilities of Geminicited by 90
8 more
  1. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  2. 2020Towards Learning a Universal Non-Semantic Representation of Speechcited by 84
  3. 2022VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captionerscited by 70
  4. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoderscited by 50
  5. 2024HeAR -- Health Acoustic Representationscited by 15
  6. 2020FRILL: A Non-Semantic Speech Embedding for Mobile Devicescited by 13
  7. 2023Domain-specific optimization and diverse evaluation of self-supervised models for histopathologycited by 11
  8. 2024Closing the AI generalization gap by adjusting for dermatology condition distribution differences across clinical settingscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.