mlsanityData to 7 Oct 2026

ECGBERT: Understanding Hidden Language of ECGs with Self-Supervised Representation Learning

arXiv 2306.06340 ↗First posted 10 Jun 2023eess.SP cs.LG q-bio.QMcited by 10

Not on the map. Why?

Abstract (arXiv)

In the medical field, current ECG signal analysis approaches rely on supervised deep neural networks trained for specific tasks that require substantial amounts of labeled data. However, our paper introduces ECGBERT, a self-supervised representation learning approach that unlocks the underlying language of ECGs. By unsupervised pre-training of the model, we mitigate challenges posed by the lack of well-labeled and curated medical data. ECGBERT, inspired by advances in the area of natural language processing and large language models, can be fine-tuned with minimal additional layers for various ECG-based problems. Through four tasks, including Atrial Fibrillation arrhythmia detection, heartbeat classification, sleep apnea detection, and user authentication, we demonstrate ECGBERT's potential to achieve state-of-the-art results on a wide variety of tasks.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 21 of its 57 references that are in the map. First posted on arXiv 10 Jun 2023.

Builds on

21 of 57 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  6. 2021Transformers in Vision: A Surveycited by 1,009
  7. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  8. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
  9. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  10. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  11. 2020Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XLcited by 158
  12. 2018ECG Heartbeat Classification: A Deep Transferable Representationcited by 76
9 more
  1. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  2. 2021Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processingcited by 57
  3. 2017Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networkscited by 52
  4. 2023Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models?cited by 21
  5. 2018Inter- and intra- patient ECG heartbeat classification for arrhythmia detection: a sequence to sequence deep learning approachcited by 15
  6. 2020HAN-ECG: An Interpretable Atrial Fibrillation Detection Model Using Hierarchical Attention Networkscited by 14
  7. 2023Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standardscited by 8
  8. 2018ECG Arrhythmia Classification Using Transfer Learning from 2-Dimensional Deep CNN Featurescited by 7
  9. 2020ECG Language Processing (ELP): a New Technique to Analyze ECG Signalscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 2.

10 papers.

  1. 2024ECG-FM: An Open Electrocardiogram Foundation Modelcited by 89
  2. 2025Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Modelcited by 36
  3. 2024Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenizationcited by 26
  4. 2024ECG-Byte: A Tokenizer for End-to-End Generative Electrocardiogram Language Modelingcited by 9
  5. 2025Generative AI for Healthcare: Fundamentals, Challenges, and Perspectivescited by 3
  6. 2026RhythmBERT: A Self-Supervised Language Model Based on Latent Representations of ECG Waveforms for Heart Disease Detectioncited by 3
  7. 2024LiPCoT: Linear Predictive Coding based Tokenizer for Self-supervised Learning of Time Series Data via Language Modelscited by 2
  8. 2024HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysiscited by 2
  9. 2025Generalist vs Specialist Time Series Foundation Models: Investigating Potential Emergent Behaviors in Assessing Human Health Using PPG Signalscited by 0
  10. 2026Large language models in medical time series analysiscited by 0