mlsanityData to 13 Oct 2026

HeAR -- Health Acoustic Representations

arXiv 2403.02522 ↗First posted 4 Mar 2024cs.LG cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Health acoustic sounds such as coughs and breaths are known to contain useful health signals with significant potential for monitoring health and disease, yet are underexplored in the medical machine learning community. The existing deep learning systems for health acoustics are often narrowly trained and evaluated on a single task, which is limited by data and may hinder generalization to other tasks. To mitigate these gaps, we develop HeAR, a scalable self-supervised learning-based deep learning system using masked autoencoders trained on a large dataset of 313 million two-second long audio clips. Through linear probes, we establish HeAR as a state-of-the-art health audio embedding model on a benchmark of 33 health acoustic tasks across 6 datasets. By introducing this work, we hope to enable and accelerate further health acoustics research.

Key facts. Cited by 15 papers in the map (data through 13 Oct 2026). Builds on 38 of its 57 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

38 of 57 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,180
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,712
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,730
  11. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,870
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,817
26 more
  1. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,488
  2. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,267
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 814
  4. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 489
  5. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 340
  6. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 322
  7. 2022Masked Autoencoders that Listencited by 272
  8. 2023A Cookbook of Self-Supervised Learningcited by 249
  9. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 229
  10. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 159
  11. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  12. 2020Towards Learning a Universal Non-Semantic Representation of Speechcited by 84
  13. 2020The COUGHVID crowdsourcing dataset: A corpus for the study of large-scale cough analysis algorithmscited by 59
  14. 2021Slow-Fast Auditory Streams For Audio Recognitioncited by 56
  15. 2021Towards Learning Universal Audio Representationscited by 51
  16. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoderscited by 50
  17. 2023Multimodal LLMs for health grounded in individual-specific datacited by 48
  18. 2016Trainable Frontend For Robust and Far-Field Keyword Spottingcited by 33
  19. 2021Universal Paralinguistic Speech Representations Using Self-Supervised Conformerscited by 32
  20. 2021Conformer-Based Self-Supervised Learning for Non-Speech Audio Taskscited by 22
  21. 2023Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infectioncited by 22
  22. 2021Automatic Cough Classification for Tuberculosis Screening in a Real-World Environmentcited by 17
  23. 2022TRILLsson: Distilled Universal Paralinguistic Speech Representationscited by 16
  24. 2020FRILL: A Non-Semantic Speech Embedding for Mobile Devicescited by 13
  25. 2020Detecting COVID-19 from Breathing and Coughing Sounds using Deep Neural Networkscited by 13
  26. 2020Can Machine Learning Be Used to Recognize and Diagnose Coughs?cited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 6.

15 papers.

Sort:

  1. 2024Scaling Wearable Foundation Modelscited by 59
  2. 2024Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarkingcited by 23
  3. 2024Health AI Developer Foundationscited by 4
  4. 2025Foundation Model Hidden Representations for Heart Rate Estimation from Auscultationcited by 1
  5. 2025XAI-Driven Spectral Analysis of Cough Sounds for Respiratory Disease Characterizationcited by 1
  6. 2025Pediatric Asthma Detection with Googles HeAR Model: An AI-Driven Respiratory Sound Classifiercited by 0
  7. 2025Automatic Cough Analysis for Non-Small Cell Lung Cancer Detectioncited by 0
  8. 2025Estimating Respiratory Effort from Nocturnal Breathing Sounds for Obstructive Sleep Apnoea Screeningcited by 0
  9. 2025AI-Driven Acoustic Voice Biomarker-Based Hierarchical Classification of Benign Laryngeal Voice Disorders from Sustained Vowelscited by 0
  10. 2026Resp-Agent: An Agent-Based System for Multimodal Respiratory Sound Generation and Disease Diagnosiscited by 0
  11. 2026Patient-level validation of a foundation-model pipeline for pediatric lung sounds: physician-annotated adventitious events are recognized, disease groups are not reliably predictedcited by 0
  12. 2026Beyond Classification: A Cough Regression Benchmark for Respiratory Acoustic Foundation Modelscited by 0
3 more
  1. 2026Unlocking In-Context Learning in Audio-Language Models from Decentralized Medical Audiocited by 0
  2. 2026BCoughBench: Benchmarking Respiratory Acoustic Foundation Models Under Body-Coupled Wearable Sensor Conditionscited by 0
  3. 2026Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screeningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Synthetic Data Augmentation for Medical Audio Classification: A Preliminary Evaluation
  2. 2026From Sound to Symptom: Real-Time Respiratory Signal Understanding for Conversational Healthcare Agents
  3. 2026StethoLM: Audio Language Model for Cardiopulmonary Analysis Across Clinical Tasks
  4. 2025Language Models as Semantic Teachers: Post-Training Alignment for Medical Audio Understanding
  5. 2024RespLLM: Unifying Audio and Text with Multimodal LLMs for Generalized Respiratory Health Prediction
  6. 2025CaReAQA: A Cardiac and Respiratory Audio Question Answering Model for Open-Ended Diagnostic Reasoning
  7. 2026Adaptive Test-Time Scaling for Zero-Shot Respiratory Audio Classification
  8. 2026Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment