# Self-supervised Normality Learning and Divergence Vector-guided Model Merging for Zero-shot Congenital Heart Disease Detection in Fetal Ultrasound Videos

arXiv: https://arxiv.org/abs/2503.07799 · HTML: https://mlsanity.com/paper/2503.07799/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Builds on (most cited first, 12 of 12)

- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [TIES-Merging: Resolving Interference When Merging Models](https://mlsanity.com/paper/2306.01708/) (2023)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [Model Stock: All we need is just a few fine-tuned models](https://mlsanity.com/paper/2403.19522/) (2024)
- [Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learning](https://mlsanity.com/paper/2212.03229/) (2022)
- [Deep Anomaly Discovery From Unlabeled Videos via Normality Advantage and Self-Paced Refinement](https://mlsanity.com/paper/2108.01975/) (2021)
- [TimeFormer: Capturing Temporal Relationships of Deformable 3D Gaussians for Robust Reconstruction](https://mlsanity.com/paper/2411.11941/) (2024)
- [VideoPatchCore: An Effective Method to Memorize Normality for Video Anomaly Detection](https://mlsanity.com/paper/2409.16225/) (2024)
- [LGN-Net: Local-Global Normality Network for Video Anomaly Detection](https://mlsanity.com/paper/2211.07454/) (2022)

## Built on by (most cited first, 2 of 2)

- [Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation](https://mlsanity.com/paper/2506.11777/) (2025)
- [TPA: Temporal Prompt Alignment for Fetal Congenital Heart Defect Classification](https://mlsanity.com/paper/2508.15298/) (2025)

## Abstract (arXiv)

Congenital Heart Disease (CHD) is one of the leading causes of fetal mortality, yet the scarcity of labeled CHD data and strict privacy regulations surrounding fetal ultrasound (US) imaging present significant challenges for the development of deep learning-based models for CHD detection. Centralised collection of large real-world datasets for rare conditions, such as CHD, from large populations requires significant co-ordination and resource. In addition, data governance rules increasingly prevent data sharing between sites. To address these challenges, we introduce, for the first time, a novel privacy-preserving, zero-shot CHD detection framework that formulates CHD detection as a normality modeling problem integrated with model merging. In our framework dubbed Sparse Tube Ultrasound Distillation (STUD), each hospital site first trains a sparse video tube-based self-supervised video anomaly detection (VAD) model on normal fetal heart US clips with self-distillation loss. This enables site-specific models to independently learn the distribution of healthy cases. To aggregate knowledge across the decentralized models while maintaining privacy, we propose a Divergence Vector-Guided Model Merging approach, DivMerge, that combines site-specific models into a single VAD model without data exchange. Our approach preserves domain-agnostic rich spatio-temporal representations, ensuring generalization to unseen CHD cases. We evaluated our approach on real-world fetal US data collected from 5 hospital sites. Our merged model outperformed site-specific models by 23.77% and 30.13% in accuracy and F1-score respectively on external test sets.