mlsanityData to 7 Oct 2026

Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners

arXiv 2410.02131 ↗First posted 3 Oct 2024cs.LG cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

The accurate interpretation of Electrocardiogram (ECG) signals is pivotal for diagnosing cardiovascular diseases. Integrating ECG signals with accompanying textual reports further holds immense potential to enhance clinical diagnostics by combining physiological data and qualitative insights. However, this integration faces significant challenges due to inherent modality disparities and the scarcity of labeled data for robust cross-modal learning. To address these obstacles, we propose D-BETA, a novel framework that pre-trains ECG and text data using a contrastive masked auto-encoder architecture. D-BETA uniquely combines the strengths of generative with boosted discriminative capabilities to achieve robust cross-modal representations. This is accomplished through masked modality modeling, specialized loss functions, and an improved negative sampling strategy tailored for cross-modal alignment. Extensive experiments on five public datasets across diverse downstream tasks demonstrate that D-BETA significantly outperforms existing methods, achieving an average AUC improvement of 15% in linear probing with only one percent of training data and 2% in zero-shot performance without requiring training data over state-of-the-art models. These results highlight the effectiveness of D-BETA, underscoring its potential to advance automated clinical diagnostics through multi-modal representations. Our sample code and checkpoint are made available at https://github.com/manhph2211/D-BETA.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

37 of 51 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  12. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
25 more
  1. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  2. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  3. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
  4. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 246
  5. 2020Self-supervised Co-training for Video Representation Learningcited by 232
  6. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  7. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  8. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 178
  9. 2020CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patientscited by 170
  10. 2019Automatic diagnosis of the 12-lead ECG using a deep neural networkcited by 164
  11. 2022Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Trainingcited by 104
  12. 2019Multi-task Self-Supervised Learning for Human Activity Detectioncited by 98
  13. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  14. 2024ECG-FM: An Open Electrocardiogram Foundation Modelcited by 89
  15. 2024Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogramcited by 84
  16. 2024Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancementcited by 66
  17. 2023Frozen Language Model Helps ECG Zero-Shot Learningcited by 62
  18. 2022Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformercited by 47
  19. 2022Lead-agnostic Self-supervised Learning for Local and Global Representations of Electrocardiogramcited by 38
  20. 20213KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentationscited by 36
  21. 2023ETP: Learning Transferable ECG Representations via ECG-Text Pre-trainingcited by 28
  22. 2024ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Textcited by 25
  23. 2022Negative Sampling for Contrastive Representation Learning: A Reviewcited by 23
  24. 2022Self-Knowledge Distillation based Self-Supervised Learning for Covid-19 Detection from Chest X-Ray Imagescited by 13
  25. 2021Hybrid Generative-Contrastive Representation Learningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

  1. 2025Interpretable Multimodal Zero-Shot ECG Diagnosis via Structured Clinical Knowledge Alignmentcited by 3
  2. 2026CardioState-JEPA: Delay-Aware Cross-Modal Learning of a Shared Cardiac Representationcited by 3
  3. 2025FOCAL: Fine-Grained Optimal-Transport-Driven Contrastive Alignment of Language and ECGs with Waveform Enhancementcited by 2
  4. 2025ECG-aBcDe: Overcoming Model Dependence, Encoding ECG into a Universal Language for Any LLMcited by 1
  5. 2026UniPACT: A Multimodal Framework for Prognostic Question Answering on Raw ECG and Structured EHRcited by 1
  6. 2026Compact Latent Manifold Translation: A Parameter-Efficient Foundation Model for Cross-Modal and Cross-Frequency Physiological Signal Synthesiscited by 1
  7. 2026LVCG: Learning ECG Representations in the Latent Vectorcardiogram Spacecited by 1
  8. 2026Position: Evaluation of ECG Representations Must Be Fixedcited by 0
  9. 2026SignalMC-MED: A Multimodal Benchmark for Evaluating Biosignal Foundation Models on Single-Lead ECG and PPGcited by 0
  10. 2026Learning ECG Image Representations via Dual Physiological-Aware Alignmentscited by 0
  11. 2026Reasoning Before Diagnosis: Physician-Inspired Structured Thinking for ECG Classificationcited by 0
  12. 2026Information-theoretic Multimodal Representation Learning for Electrocardiogram Signalscited by 0
4 more
  1. 2026EchoBridge: Long-Tail-Aware ECG-Echocardiography Text Alignment for Echocardiography-Derived Cardiac Findingscited by 0
  2. 2026ECG-LENS: Lead-Aware Clinical Context Enriched ECG Report Generation and Evaluationcited by 0
  3. 2026TRACE: Tractable Routing Autoencoder for Clinical ECGcited by 0
  4. 2026TRACE: Expert-Aligned ECG Representation Learning with Rigorous Benchmarking and Real-World Validation in Acute Cardiac Carecited by 0