# Exploiting Unlabeled Structures through Task Consistency Training for Versatile Medical Image Segmentation

arXiv: https://arxiv.org/abs/2509.04732 · HTML: https://mlsanity.com/paper/2509.04732/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Semi-Supervised Semantic Segmentation with Cross-Consistency Training](https://mlsanity.com/paper/2003.09005/) (2020)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [Semi-supervised Medical Image Segmentation through Dual-task Consistency](https://mlsanity.com/paper/2009.04448/) (2020)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [Robust Learning Through Cross-Task Consistency](https://mlsanity.com/paper/2006.04096/) (2020)
- [DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets](https://mlsanity.com/paper/2011.10217/) (2020)
- [Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation](https://mlsanity.com/paper/1904.06346/) (2019)
- [Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction](https://mlsanity.com/paper/2001.00208/) (2020)
- [UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learner](https://mlsanity.com/paper/2304.03493/) (2023)
- [MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation](https://mlsanity.com/paper/2303.14444/) (2023)
- [Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation](https://mlsanity.com/paper/2306.02416/) (2023)
- [Multi-organ Segmentation via Co-training Weight-averaged Models from Few-organ Datasets](https://mlsanity.com/paper/2008.07149/) (2020)
- [COSST: Multi-organ Segmentation with Partially Labeled Datasets Using Comprehensive Supervisions and Self-training](https://mlsanity.com/paper/2304.14030/) (2023)
- [Versatile Medical Image Segmentation Learned from Multi-Source Datasets via Model Self-Disambiguation](https://mlsanity.com/paper/2311.10696/) (2023)
- [Universal Segmentation of 33 Anatomies](https://mlsanity.com/paper/2203.02098/) (2022)
- [MS-KD: Multi-Organ Segmentation with Multiple Binary-Labeled Datasets](https://mlsanity.com/paper/2108.02559/) (2021)

## Abstract (arXiv)

Versatile medical image segmentation (VMIS) targets the segmentation of multiple classes, while obtaining full annotations for all classes is often impractical due to the time and labor required. Leveraging partially labeled datasets (PLDs) presents a promising alternative; however, current VMIS approaches face significant class imbalance due to the unequal category distribution in PLDs. Existing methods attempt to address this by generating pseudo-full labels. Nevertheless, these typically require additional models and often result in potential performance degradation from label noise. In this work, we introduce a Task Consistency Training (TCT) framework to address class imbalance without requiring extra models. TCT includes a backbone network with a main segmentation head (MSH) for multi-channel predictions and multiple auxiliary task heads (ATHs) for task-specific predictions. By enforcing a consistency constraint between the MSH and ATH predictions, TCT effectively utilizes unlabeled anatomical structures. To avoid error propagation from low-consistency, potentially noisy data, we propose a filtering strategy to exclude such data. Additionally, we introduce a unified auxiliary uncertainty-weighted loss (UAUWL) to mitigate segmentation quality declines caused by the dominance of specific tasks. Extensive experiments on eight abdominal datasets from diverse clinical sites demonstrate our approach's effectiveness.