# DINO-Mix: Distilling Foundational Knowledge with Cross-Domain CutMix for Semi-supervised Class-imbalanced Medical Image Segmentation

arXiv: https://arxiv.org/abs/2602.07819 · HTML: https://mlsanity.com/paper/2602.07819/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 8 Feb 2026.

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [Semi-supervised Medical Image Segmentation through Dual-task Consistency](https://mlsanity.com/paper/2009.04448/) (2020)
- [U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation](https://mlsanity.com/paper/2406.02918/) (2024)
- [CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2102.09559/) (2021)
- [Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency](https://mlsanity.com/paper/2012.07042/) (2020)
- [ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning](https://mlsanity.com/paper/2110.10368/) (2021)
- [SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance](https://mlsanity.com/paper/2311.16241/) (2023)
- [Calibrating Label Distribution for Class-Imbalanced Barely-Supervised Knee Segmentation](https://mlsanity.com/paper/2205.03644/) (2022)
- [Addressing Class Imbalance in Semi-supervised Image Segmentation: A Study on Cardiac MRI](https://mlsanity.com/paper/2209.00123/) (2022)
- [Semi-supervised 3D Semantic Scene Completion with 2D Vision Foundation Model Guidance](https://mlsanity.com/paper/2408.11559/) (2024)
- [Semantic Consistency Regularization with Large Language Models for Semi-supervised Sentiment Analysis](https://mlsanity.com/paper/2501.17598/) (2025)

## Abstract (arXiv)

Semi-supervised learning (SSL) has emerged as a critical paradigm for medical image segmentation, mitigating the immense cost of dense annotations. However, prevailing SSL frameworks are fundamentally "inward-looking", recycling information and biases solely from within the target dataset. This design triggers a vicious cycle of confirmation bias under class imbalance, leading to the catastrophic failure to recognize minority classes. To dismantle this systemic issue, we propose a paradigm shift to a multi-level "outward-looking" framework. Our primary innovation is Foundational Knowledge Distillation (FKD), which looks outward beyond the confines of medical imaging by introducing a pre-trained visual foundation model, DINOv3, as an unbiased external semantic teacher. Instead of trusting the student's biased high confidence, our method distills knowledge from DINOv3's robust understanding of high semantic uniqueness, providing a stable, cross-domain supervisory signal that anchors the learning of minority classes. To complement this core strategy, we further look outward within the data by proposing Progressive Imbalance-aware CutMix (PIC), which creates a dynamic curriculum that adaptively forces the model to focus on minority classes in both labeled and unlabeled subsets. This layered strategy forms our framework, DINO-Mix, which breaks the vicious cycle of bias and achieves remarkable performance on challenging semi-supervised class-imbalanced medical image segmentation benchmarks Synapse and AMOS.