# KA2ER: Knowledge Adaptive Amalgamation of ExpeRts for Medical Images Segmentation

arXiv: https://arxiv.org/abs/2410.21085 · HTML: https://mlsanity.com/paper/2410.21085/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Multivariate mixture model for myocardium segmentation combining multi-source images](https://mlsanity.com/paper/1612.08820/) (2016)
- [Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge](https://mlsanity.com/paper/1902.07880/) (2019)
- [SegVol: Universal and Interactive Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2311.13385/) (2023)
- [From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels](https://mlsanity.com/paper/2303.13005/) (2023)
- [TotalSegmentator MRI: Robust Sequence-independent Segmentation of Multiple Anatomic Structures in MRI](https://mlsanity.com/paper/2405.19492/) (2024)
- [BayeSeg: Bayesian Modeling for Medical Image Segmentation with Interpretable Generalizability](https://mlsanity.com/paper/2303.01710/) (2023)
- [Amalgamating Knowledge towards Comprehensive Classification](https://mlsanity.com/paper/1811.02796/) (2018)
- [Medical Image Analysis on Left Atrial LGE MRI for Atrial Fibrillation Studies: A Review](https://mlsanity.com/paper/2106.09862/) (2021)
- [AtrialJSQnet: A New Framework for Joint Segmentation and Quantification of Left Atrium and Scars Incorporating Spatial and Shape Information](https://mlsanity.com/paper/2008.04729/) (2020)
- [MyoPS-Net: Myocardial Pathology Segmentation with Flexible Combination of Multi-Sequence CMR Images](https://mlsanity.com/paper/2211.03062/) (2022)
- [AtrialGeneral: Domain Generalization for Left Atrial Segmentation of Multi-Center LGE MRIs](https://mlsanity.com/paper/2106.08727/) (2021)
- [MERIT: Multi-view evidential learning for reliable and interpretable liver fibrosis staging](https://mlsanity.com/paper/2405.02918/) (2024)
- [Contrastive Knowledge Amalgamation for Unsupervised Image Classification](https://mlsanity.com/paper/2307.14781/) (2023)
- [Knowledge Amalgamation for Object Detection with Transformers](https://mlsanity.com/paper/2203.03187/) (2022)
- [Joint Modeling of Image and Label Statistics for Enhancing Model Generalizability of Medical Image Segmentation](https://mlsanity.com/paper/2206.04336/) (2022)

## Built on by (most cited first, 1 of 1)

- [Uncertainty-Aware Multi-Expert Knowledge Distillation for Imbalanced Disease Grading](https://mlsanity.com/paper/2505.00592/) (2025)

## Abstract (arXiv)

Recently, many foundation models for medical image analysis such as MedSAM, SwinUNETR have been released and proven to be useful in multiple tasks. However, considering the inherent heterogeneity and inhomogeneity of real-world medical data, directly applying these models to specific medical image segmentation tasks often leads to negative domain shift effects, which can severely weaken the model's segmentation capabilities. To this end, we propose an adaptive amalgamation knowledge framework that aims to train a versatile foundation model to handle the joint goals of multiple expert models, each specialized for a distinct task. Specifically, we first train an nnUNet-based expert model for each task, and reuse the pre-trained SwinUNTER as the target foundation model. Then, the input data for all challenging tasks are encoded in the foundation model and the expert models, respectively, and their backbone features are jointly projected into the adaptive amalgamation layer. Within the hidden layer, the hierarchical attention mechanisms are designed to achieve adaptive merging of the target model to the hidden layer feature knowledge of all experts, which significantly reduces the domain shift arising from the inter-task differences. Finally, the gold amalgamated features and the prompt features are fed into the mask decoder to obtain the segmentation results. Extensive experiments conducted in these challenging tasks demonstrate the effectiveness and adaptability of our foundation model for real-world medical image segmentation.