# Scattering Transformer: A Training-Free Transformer Architecture for Heart Murmur Detection

arXiv: https://arxiv.org/abs/2509.18424 · HTML: https://mlsanity.com/paper/2509.18424/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 9 of 9)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [The CirCor DigiScope Dataset: From Murmur Detection to Murmur Classification](https://mlsanity.com/paper/2108.00813/) (2021)
- [Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input](https://mlsanity.com/paper/2210.14648/) (2022)
- [A Comprehensive Survey on Heart Sound Analysis in the Deep Learning Era](https://mlsanity.com/paper/2301.09362/) (2023)
- [ENACT-Heart -- ENsemble-based Assessment Using CNN and Transformer on Heart Sounds](https://mlsanity.com/paper/2502.16914/) (2025)
- [Exploring Pre-trained General-purpose Audio Representations for Heart Murmur Detection](https://mlsanity.com/paper/2404.17107/) (2024)

## Built on by (most cited first, 1 of 1)

- [Manifold-Aware Diffusion-Augmented Contrastive Learning for Noise-Robust Biosignal Representation](https://mlsanity.com/paper/2509.20048/) (2025)

## Abstract (arXiv)

In an attempt to address the need for skilled clinicians in heart sound interpretation, recent research efforts on automating cardiac auscultation have explored deep learning approaches. The majority of these approaches have been based on supervised learning that is always challenged in occasions where training data is limited. More recently, there has been a growing interest in potentials of pre-trained self-supervised audio foundation models for biomedical end tasks. Despite exhibiting promising results, these foundational models are typically computationally intensive. Within the context of automatic cardiac auscultation, this study explores a lightweight alternative to these general-purpose audio foundation models by introducing the Scattering Transformer, a novel, training-free transformer architecture for heart murmur detection. The proposed method leverages standard wavelet scattering networks by introducing contextual dependencies in a transformer-like architecture without any backpropagation. We evaluate our approach on the public CirCor DigiScope dataset, directly comparing it against leading general-purpose foundational models. The Scattering Transformer achieves a Weighted Accuracy(WAR) of 0.786 and an Unweighted Average Recall(UAR) of 0.697, demonstrating performance highly competitive with contemporary state of the art methods. This study establishes the Scattering Transformer as a viable and promising alternative in resource-constrained setups.