# Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment

arXiv: https://arxiv.org/abs/2605.15720 · HTML: https://mlsanity.com/paper/2605.15720/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 15 May 2026.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [Learning with Pseudo-Ensembles](https://mlsanity.com/paper/1412.4864/) (2014)
- [Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels](https://mlsanity.com/paper/2203.03884/) (2022)
- [ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning](https://mlsanity.com/paper/2007.07936/) (2020)
- [LViT: Language meets Vision Transformer in Medical Image Segmentation](https://mlsanity.com/paper/2206.14718/) (2022)
- [MosMedData: Chest CT Scans With COVID-19 Related Findings Dataset](https://mlsanity.com/paper/2005.06465/) (2020)
- [Unbiased Teacher v2: Semi-supervised Object Detection for Anchor-free and Anchor-based Detectors](https://mlsanity.com/paper/2206.09500/) (2022)
- [MixGen: A New Multi-Modal Data Augmentation](https://mlsanity.com/paper/2206.08358/) (2022)
- [OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Images](https://mlsanity.com/paper/2202.10185/) (2022)
- [Active Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2303.08348/) (2023)
- [Ariadne's Thread:Using Text Prompts to Improve Segmentation of Infected Areas from Chest X-ray images](https://mlsanity.com/paper/2307.03942/) (2023)
- [SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation](https://mlsanity.com/paper/2406.01451/) (2024)
- [RESMatch: Referring Expression Segmentation in a Semi-Supervised Manner](https://mlsanity.com/paper/2402.05589/) (2024)
- [SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidance](https://mlsanity.com/paper/2409.04758/) (2024)

## Abstract (arXiv)

Medical referring image segmentation (MRIS) predicts lesion masks from medical images and natural-language referring expressions, but acquiring paired pixel-level annotations and referring texts is costly. Semi-supervised learning (SSL) can alleviate this burden by exploiting unlabeled data, yet its effectiveness depends on preserving image--text alignment under strong perturbations. Existing SSL methods for referring segmentation rely on independent or simple multimodal perturbations (e.g., left--right flips), while stronger augmentations such as CutMix remain largely unexplored because they can disrupt cross-modal correspondence. We propose Semi-MedRef, a teacher--student SSL framework that explicitly preserves alignment between medical images and positional language through three complementary components: T-PatchMix, an alignment-preserving cross-modal augmentation that synchronizes patch mixing with positional-language and pseudo-mask updates; PosAug, a position-aware text augmentation that regularizes reliance on positional expressions; and Positional Affinity Contrastive Learning (PACL), which exploits coarse positional cues to construct region-aware supervision through anatomically weighted soft positives, encouraging anatomically grounded cross-modal representation learning. Experiments on QaTa-COV19 and MosMedData+ demonstrate that Semi-MedRef consistently outperforms state-of-the-art fully supervised and semi-supervised MRIS methods across all label regimes.