# CoMLP: Cooperatively-Gated MLPs for Fine-Grained Cross-Modal Information Fusion in Medical Image Segmentation

arXiv: https://arxiv.org/abs/2609.04781 · HTML: https://mlsanity.com/paper/2609.04781/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 4 Sep 2026.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [LViT: Language meets Vision Transformer in Medical Image Segmentation](https://mlsanity.com/paper/2206.14718/) (2022)
- [MosMedData: Chest CT Scans With COVID-19 Related Findings Dataset](https://mlsanity.com/paper/2005.06465/) (2020)
- [U-Net Transformer: Self and Cross Attention for Medical Image Segmentation](https://mlsanity.com/paper/2103.06104/) (2021)
- [Improved Transformer for High-Resolution GANs](https://mlsanity.com/paper/2106.07631/) (2021)
- [OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Images](https://mlsanity.com/paper/2202.10185/) (2022)
- [Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis](https://mlsanity.com/paper/2503.20047/) (2025)
- [Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration](https://mlsanity.com/paper/2406.00123/) (2024)
- [Ariadne's Thread:Using Text Prompts to Improve Segmentation of Infected Areas from Chest X-ray images](https://mlsanity.com/paper/2307.03942/) (2023)
- [Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models](https://mlsanity.com/paper/2308.07706/) (2023)
- [SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidance](https://mlsanity.com/paper/2409.04758/) (2024)
- [Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer](https://mlsanity.com/paper/2307.03427/) (2023)
- [3DPX: Progressive 2D-to-3D Oral Image Reconstruction with Hybrid MLP-CNN Networks](https://mlsanity.com/paper/2408.01292/) (2024)
- [MLP Architectures for Vision-and-Language Modeling: An Empirical Study](https://mlsanity.com/paper/2112.04453/) (2021)
- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)
- [Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignments](https://mlsanity.com/paper/2412.13533/) (2024)
- [Alleviating Textual Reliance in Medical Language-guided Segmentation via Prototype-driven Semantic Approximation](https://mlsanity.com/paper/2507.11055/) (2025)
- [SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images](https://mlsanity.com/paper/2302.03861/) (2023)
- [MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentation](https://mlsanity.com/paper/2601.09879/) (2026)

## Abstract (arXiv)

Multi-modal medical images and clinical reports provide complementary anatomical, functional, and semantic information for medical image segmentation. Effectively exploiting these heterogeneous sources requires fine-grained cross-modal information fusion that preserves subtle spatial details while capturing semantic dependencies across modalities. Existing fusion approaches frequently rely on cross-attention, whose computational burden increases rapidly with spatial resolution, making dense cross-modal interaction difficult on high-resolution feature maps, particularly for volumetric medical images. In this work, we propose CoMLP, a cooperatively-gated MLP module for fine-grained cross-modal information fusion in medical image segmentation. CoMLP models cross-modal dependencies through cooperative cross-gating, built upon complementary regional and dilated MLP interactions, to capture local and global cross-modal dependencies. We further develop a multi-source fusion architecture in which CoMLP performs both inter-image fusion across imaging modalities and vision-language fusion between visual features and textual reports, enabling heterogeneous information to be integrated without relying on dense cross-attention. Extensive experiments on five medical segmentation benchmarks, covering 2D/3D images, clinical reports, multiple imaging modalities, and diverse anatomical regions, demonstrate consistent improvements over state-of-the-art multi-modal and language-guided segmentation methods. Ablation studies further show that fine-grained interaction at high spatial resolutions and complementary local-global fusion are critical to the performance gains. These results demonstrate the potential of MLP-based interaction as an effective alternative for fine-grained cross-modal information fusion in medical image segmentation.