# LSTrans: Efficient Knowledge Transfer for Lightweight and Automated ECG Classification

arXiv: https://arxiv.org/abs/2607.10784 · HTML: https://mlsanity.com/paper/2607.10784/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 12 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Automatic diagnosis of the 12-lead ECG using a deep neural network](https://mlsanity.com/paper/1904.01949/) (2019)
- [Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL](https://mlsanity.com/paper/2004.13701/) (2020)
- [An Electrocardiogram Foundation Model Built on over 10 Million Recordings with External Evaluation across Multiple Domains](https://mlsanity.com/paper/2410.04133/) (2024)
- [ECG-Image-Kit: A Synthetic Image Generation Toolbox to Facilitate Deep Learning-Based Electrocardiogram Digitization](https://mlsanity.com/paper/2307.01946/) (2023)
- [CE-SSL: Computation-Efficient Semi-Supervised Learning for ECG-based Cardiovascular Diseases Detection](https://mlsanity.com/paper/2406.14377/) (2024)
- [DiReDi: Distillation and Reverse Distillation for AIoT Applications](https://mlsanity.com/paper/2409.08308/) (2024)
- [High-Accuracy ECG Image Interpretation using Parameter-Efficient LoRA Fine-Tuning with Multimodal LLaMA 3.2](https://mlsanity.com/paper/2501.18670/) (2025)

## Abstract (arXiv)

Deploying deep learning models for automated electrocardiogram classification on resource-constrained wearable devices remains challenging due to high computational costs. To address this, we propose LSTrans, a lightweight hybrid model designed for efficient and sensitive ECG analysis. LSTrans introduces a specialized 1D convolutional backbone with an interleaved layer architecture to capture both macroscopic rhythmic trends and microscopic morphological variations. This backbone is cascaded with a Transformer encoder to model long-range temporal dependencies, incorporating Low-Rank Adaptation across critical layers to compress the model and reduce the trainable parameter space. We further employ homogeneous and heterogeneous knowledge distillation to transfer diagnostic expertise from high-capacity teacher models to the student. Experimental results on multiple benchmark datasets demonstrate that LSTrans achieves a competitive balance between diagnostic sensitivity and resource efficiency, substantially reducing peak memory footprints and training latency during downstream adaptation. The source code is available for review at https://github.com/zyee00128/LSTrans4BIBM.