# Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention

arXiv: https://arxiv.org/abs/2405.06284 · HTML: https://mlsanity.com/paper/2405.06284/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 72 references that are in the map.
- First posted on arXiv 10 May 2024.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [ResNeSt: Split-Attention Networks](https://mlsanity.com/paper/2004.08955/) (2020)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs](https://mlsanity.com/paper/1910.03667/) (2019)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [CE-Net: Context Encoder Network for 2D Medical Image Segmentation](https://mlsanity.com/paper/1903.02740/) (2019)
- [ResUNet++: An Advanced Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1911.07067/) (2019)
- [Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1605.01397/) (2016)
- [Learning in the Frequency Domain](https://mlsanity.com/paper/2002.12416/) (2020)
- [Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images](https://mlsanity.com/paper/2004.14133/) (2020)
- [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020)

## Built on by (most cited first, 26 of 26)

- [SimMLM: A Simple Framework for Multi-modal Learning with Missing Modality](https://mlsanity.com/paper/2507.19264/) (2025)
- [De-LightSAM: Modality-Decoupled Lightweight SAM for Generalizable Medical Segmentation](https://mlsanity.com/paper/2407.14153/) (2024)
- [M2SFormer: Multi-Spectral and Multi-Scale Attention with Edge-Aware Difficulty Guidance for Image Forgery Localization](https://mlsanity.com/paper/2506.20922/) (2025)
- [UniUltra: Interactive Parameter-Efficient SAM2 for Universal Ultrasound Segmentation](https://mlsanity.com/paper/2511.15771/) (2025)
- [Hierarchical Self-Prompting SAM: A Prompt-Free Medical Image Segmentation Framework](https://mlsanity.com/paper/2506.02854/) (2025)
- [Co-Seg: Mutual Prompt-Guided Collaborative Learning for Tissue and Nuclei Segmentation](https://mlsanity.com/paper/2509.06740/) (2025)
- [FreqDINO: Frequency-Guided Adaptation for Generalized Boundary-Aware Ultrasound Image Segmentation](https://mlsanity.com/paper/2512.11335/) (2025)
- [EEMS: Edge-Prompt Enhanced Medical Image Segmentation Based on Learnable Gating Mechanism](https://mlsanity.com/paper/2510.11287/) (2025)
- [Effective and Robust Multimodal Medical Image Analysis](https://mlsanity.com/paper/2602.15346/) (2026)
- [Similarity Memory Prior is All You Need for Medical Image Segmentation](https://mlsanity.com/paper/2507.00585/) (2025)
- [LGMSNet: Thinning a medical image segmentation model via dual-level multiscale fusion](https://mlsanity.com/paper/2508.15476/) (2025)
- [Dual-supervised Asymmetric Co-training for Semi-supervised Medical Domain Generalization](https://mlsanity.com/paper/2509.20785/) (2025)
- [RobustMedSAM: Degradation-Resilient Medical Image Segmentation via Robust Foundation Model Adaptation](https://mlsanity.com/paper/2604.09814/) (2026)
- [TransGUNet: Transformer Meets Graph-based Skip Connection for Medical Image Segmentation](https://mlsanity.com/paper/2502.09931/) (2025)
- [HER-Seg: Holistically Efficient Segmentation for High-Resolution Medical Images](https://mlsanity.com/paper/2504.06205/) (2025)
- [FAD-Net: Frequency-Domain Attention-Guided Diffusion Network for Coronary Artery Segmentation using Invasive Coronary Angiography](https://mlsanity.com/paper/2506.11454/) (2025)
- [DTEA: Dynamic Topology Weaving and Instability-Driven Entropic Attenuation for Medical Image Segmentation](https://mlsanity.com/paper/2510.11259/) (2025)
- [EcoScaleNet: A Lightweight Multi Kernel Network for Long Sequence 12 lead ECG Classification](https://mlsanity.com/paper/2510.24748/) (2025)
- [Not All Pixels Are Equal: Pixel-wise Meta-Learning for Medical Segmentation with Noisy Labels](https://mlsanity.com/paper/2511.18894/) (2025)
- [HyPCA-Net: Advancing Multimodal Fusion in Medical Image Analysis](https://mlsanity.com/paper/2602.16245/) (2026)
- [AWPD: Frequency Shield Network for Agnostic Watermark Presence Detection](https://mlsanity.com/paper/2603.06723/) (2026)
- [From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation](https://mlsanity.com/paper/2604.17455/) (2026)
- [SARIF: Segment Anything for Robust Image Forensics](https://mlsanity.com/paper/2606.21108/) (2026)
- [Advancing Multimodal Fusion on Heterogeneous Medical Data with Hybrid Geometry Attention](https://mlsanity.com/paper/2607.19086/) (2026)
- [Feature Reconfiguration With Visual Prior for Medical Lesion Segmentation](https://mlsanity.com/paper/2609.03535/) (2026)
- [FreqDINO++: A Frequency-Guided Multi-Task Routing Vision Foundation Model for Universal Ultrasound Analysis](https://mlsanity.com/paper/2609.20340/) (2026)

## Abstract (arXiv)

Generalizability in deep neural networks plays a pivotal role in medical image segmentation. However, deep learning-based medical image analyses tend to overlook the importance of frequency variance, which is critical element for achieving a model that is both modality-agnostic and domain-generalizable. Additionally, various models fail to account for the potential information loss that can arise from multi-task learning under deep supervision, a factor that can impair the model representation ability. To address these challenges, we propose a Modality-agnostic Domain Generalizable Network (MADGNet) for medical image segmentation, which comprises two key components: a Multi-Frequency in Multi-Scale Attention (MFMSA) block and Ensemble Sub-Decoding Module (E-SDM). The MFMSA block refines the process of spatial feature extraction, particularly in capturing boundary features, by incorporating multi-frequency and multi-scale features, thereby offering informative cues for tissue outline and anatomical structures. Moreover, we propose E-SDM to mitigate information loss in multi-task learning with deep supervision, especially during substantial upsampling from low resolution. We evaluate the segmentation performance of MADGNet across six modalities and fifteen datasets. Through extensive experiments, we demonstrate that MADGNet consistently outperforms state-of-the-art models across various modalities, showcasing superior segmentation performance. This affirms MADGNet as a robust solution for medical image segmentation that excels in diverse imaging scenarios. Our MADGNet code is available in GitHub Link.