Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment
Not on the map. Why?
Abstract (arXiv)
Medical referring image segmentation (MRIS) predicts lesion masks from medical images and natural-language referring expressions, but acquiring paired pixel-level annotations and referring texts is costly. Semi-supervised learning (SSL) can alleviate this burden by exploiting unlabeled data, yet its effectiveness depends on preserving image--text alignment under strong perturbations. Existing SSL methods for referring segmentation rely on independent or simple multimodal perturbations (e.g., left--right flips), while stronger augmentations such as CutMix remain largely unexplored because they can disrupt cross-modal correspondence. We propose Semi-MedRef, a teacher--student SSL framework that explicitly preserves alignment between medical images and positional language through three complementary components: T-PatchMix, an alignment-preserving cross-modal augmentation that synchronizes patch mixing with positional-language and pseudo-mask updates; PosAug, a position-aware text augmentation that regularizes reliance on positional expressions; and Positional Affinity Contrastive Learning (PACL), which exploits coarse positional cues to construct region-aware supervision through anatomically weighted soft positives, encouraging anatomically grounded cross-modal representation learning. Experiments on QaTa-COV19 and MosMedData+ demonstrate that Semi-MedRef consistently outperforms state-of-the-art fully supervised and semi-supervised MRIS methods across all label regimes.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 15 May 2026.
Builds on
24 of 38 references.
References · 24
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2023Segment Anythingcited by 8,234
- 2022A ConvNet for the 2020scited by 3,579
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
- 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
- 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
- 2022MONAI: An open-source framework for deep learning in healthcarecited by 448
- 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
- 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
- 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
- 2014Learning with Pseudo-Ensemblescited by 241
12 more
- 2022Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labelscited by 218
- 2020ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learningcited by 209
- 2022LViT: Language meets Vision Transformer in Medical Image Segmentationcited by 111
- 2020MosMedData: Chest CT Scans With COVID-19 Related Findings Datasetcited by 104
- 2022Unbiased Teacher v2: Semi-supervised Object Detection for Anchor-free and Anchor-based Detectorscited by 69
- 2022MixGen: A New Multi-Modal Data Augmentationcited by 61
- 2022OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Imagescited by 48
- 2023Active Teacher for Semi-Supervised Object Detectioncited by 43
- 2023Ariadne's Thread:Using Text Prompts to Improve Segmentation of Infected Areas from Chest X-ray imagescited by 34
- 2024SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentationcited by 17
- 2024RESMatch: Referring Expression Segmentation in a Semi-Supervised Mannercited by 11
- 2024SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidancecited by 11
Built on by
Not cited yet.