# EVL-ECG: Efficient ECG Interpretation With Multi-Aspect Heterogeneous Knowledge Distillation

arXiv: https://arxiv.org/abs/2605.29977 · HTML: https://mlsanity.com/paper/2605.29977/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 28 May 2026.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 20 of 20)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [Knowledge Fusion of Large Language Models](https://mlsanity.com/paper/2401.10491/) (2024)
- [Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs](https://mlsanity.com/paper/2402.12030/) (2024)
- [ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogram](https://mlsanity.com/paper/2306.15681/) (2023)
- [Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks](https://mlsanity.com/paper/1707.01836/) (2017)
- [LLaVA-KD: A Framework of Distilling Multimodal Large Language Models](https://mlsanity.com/paper/2410.16236/) (2024)
- [GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images](https://mlsanity.com/paper/2503.06073/) (2025)
- [ECG-Chat: A Large ECG-Language Model for Cardiac Disease Diagnosis](https://mlsanity.com/paper/2408.08849/) (2024)
- [Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models](https://mlsanity.com/paper/2412.14528/) (2024)
- [MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation](https://mlsanity.com/paper/2403.04945/) (2024)
- [Teach Multimodal LLMs to Comprehend Electrocardiographic Images](https://mlsanity.com/paper/2410.19008/) (2024)
- [Dual-Space Knowledge Distillation for Large Language Models](https://mlsanity.com/paper/2406.17328/) (2024)
- [EM-KD: Distilling Efficient Multimodal Large Language Model with Unbalanced Vision Tokens](https://mlsanity.com/paper/2511.21106/) (2025)

## Abstract (arXiv)

High-fidelity ECG interpretation is increasingly reliant on massive foundation models, yet their deployment in clinical edge-care remains hindered by extreme computational demands. While knowledge distillation (KD) is a promising solution, traditional methods fail to capture the complex spatio-temporal dependencies of ECG signals when transferring knowledge across heterogeneous architectures. In this paper, we propose EVL-ECG, a framework specifically designed for cross-architecture distillation of cardiac diagnostic logic. EVL-ECG introduces three ECG-aware innovations: (1) Multi-Head Cross-Attention Alignment, which harmonizes architectural discrepancies to preserve fine-grained morphological features; (2) Optimal Transport-based Visual Feature Matching, utilizing optimal transport to maintain global structural relationships across ECG leads despite mismatched token representations; and (3) Geometric Intra-Architecture Relation Matching, which distills the latent diagnostic reasoning of the teacher model. Evaluations across ECG benchmarks demonstrate that EVL-ECG yields improvements of up to 2.4% AUC and 1.1% clinical accuracy over existing baselines. Notably, EVL-ECG establishes an efficient 2B-parameter ECG foundation model, suitable for resource-constrained clinical environments.