mlsanityData to 14 Oct 2026

CAST: Contrastive Adaptation and Distillation for Semi-Supervised Instance Segmentation

arXiv 2505.21904 ↗First posted 28 May 2025cs.CV cs.AIcited by 0

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Instance segmentation demands costly per-pixel annotations and computationally expensive models. We introduce CAST, a semi-supervised knowledge distillation (SSKD) framework that compresses pre-trained vision foundation models (VFM) into compact experts using limited labeled and abundant unlabeled data. CAST unfolds in three stages: (1) domain adaptation of the VFM(s) via self-training with contrastive calibration, (2) knowledge transfer through a unified multi-objective loss, and (3) student refinement to mitigate residual pseudo-label bias. Central to CAST is an instance-aware pixel-wise contrastive loss that fuses mask and class scores to extract informative negatives and enforce clear inter-instance margins. By maintaining this contrastive signal across both adaptation and distillation, we align teacher and student embeddings and fully leverage unlabeled images. On Cityscapes and ADE20K, our ~11x smaller student improves over its zero-shot VFM teacher(s) by +8.5 and +7.1 AP, surpasses adapted teacher(s) by +3.4 and +1.5 AP, and further outperforms state-of-the-art SSKD methods on both benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 44 of its 48 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

44 of 48 references.

References · 44

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2017Mask R-CNNcited by 7,277
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  10. 2021Vision Transformers for Dense Predictioncited by 1,803
  11. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  12. 2024Depth Anything V2cited by 1,422
32 more
  1. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  2. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  3. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  4. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  5. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  6. 2021Semi-Supervised Semantic Segmentation with Cross Pseudo Supervisioncited by 463
  7. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  8. 2021Unbiased Teacher for Semi-Supervised Object Detectioncited by 306
  9. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  10. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  11. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 216
  12. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  13. 2022Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentationcited by 178
  14. 2020Channel-wise Knowledge Distillation for Dense Predictioncited by 163
  15. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  16. 2021Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bankcited by 125
  17. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  18. 2021Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentationcited by 101
  19. 2023CLIP-KD: An Empirical Study of CLIP Model Distillationcited by 87
  20. 2024An Open and Comprehensive Pipeline for Unified Object Grounding and Detectioncited by 65
  21. 2024UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentationcited by 54
  22. 2020Self-supervised Knowledge Distillation for Few-shot Learningcited by 41
  23. 2023DIME-FM: DIstilling Multimodal and Efficient Foundation Modelscited by 28
  24. 2022ContrastMask: Contrastive Learning to Segment Every Thingcited by 20
  25. 2024Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learningcited by 19
  26. 2023Pseudo-label Alignment for Semi-supervised Instance Segmentationcited by 17
  27. 2024MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolutioncited by 17
  28. 2025Multi-Teacher Knowledge Distillation with Reinforcement Learning for Visual Recognitioncited by 16
  29. 2023Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Modelscited by 15
  30. 2023Guided Distillation for Semi-Supervised Instance Segmentationcited by 12
  31. 2024Accessing Vision Foundation Models via ImageNet-1Kcited by 10
  32. 2025S^4M: Boosting Semi-Supervised Instance Segmentation with SAMcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training a Student Expert via Semi-Supervised Foundation Model Distillation
  2. 2023Augmentation-Free Dense Contrastive Knowledge Distillation for Efficient Semantic Segmentation
  3. 2025DUDA: Distilled Unsupervised Domain Adaptation for Lightweight Semantic Segmentation
  4. 2023MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation
  5. 2023Inherit with Distillation and Evolve with Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory
  6. 2022Polite Teacher: Semi-Supervised Instance Segmentation with Mutual Learning and Pseudo-Label Thresholding
  7. 2024Better (pseudo-)labels for semi-supervised instance segmentation
  8. 2026iFAN: Inference-Aware Learning for Plain Mask Transformers