# AllSpark: Reborn Labeled Features from Unlabeled in Transformer for Semi-Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/2403.01818 · HTML: https://mlsanity.com/paper/2403.01818/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 60 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [OneFormer: One Transformer to Rule Universal Image Segmentation](https://mlsanity.com/paper/2211.06220/) (2022)
- [Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels](https://mlsanity.com/paper/2203.03884/) (2022)
- [UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer](https://mlsanity.com/paper/2109.04335/) (2021)
- [ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.05095/) (2021)

## Built on by (most cited first, 30 of 35)

- [UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2410.10777/) (2024)
- [AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation](https://mlsanity.com/paper/2409.20398/) (2024)
- [Semi-supervised Semantic Segmentation for Remote Sensing Images via Multi-scale Uncertainty Consistency and Cross-Teacher-Student Attention](https://mlsanity.com/paper/2501.10736/) (2025)
- [Tuning-free Universally-Supervised Semantic Segmentation](https://mlsanity.com/paper/2405.14294/) (2024)
- [PrevMatch: Revisiting and Maximizing Temporal Knowledge in Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2405.20610/) (2024)
- [When Confidence Fails: Revisiting Pseudo-Label Selection in Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2509.16704/) (2025)
- [ConformalSAM: Unlocking the Potential of Foundational Segmentation Models in Semi-Supervised Semantic Segmentation with Conformal Prediction](https://mlsanity.com/paper/2507.15803/) (2025)
- [Uncertainty and Energy based Loss Guided Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2501.01640/) (2025)
- [Rethinking Semi-supervised Segmentation Beyond Accuracy: Reliability and Robustness](https://mlsanity.com/paper/2506.05917/) (2025)
- [DEARLi: Decoupled Enhancement of Recognition and Localization for Semi-supervised Panoptic Segmentation](https://mlsanity.com/paper/2507.10118/) (2025)
- [S5: Scalable Semi-Supervised Semantic Segmentation in Remote Sensing](https://mlsanity.com/paper/2508.12409/) (2025)
- [Towards Robust Pseudo-Label Learning in Semantic Segmentation: An Encoding Perspective](https://mlsanity.com/paper/2512.06870/) (2025)
- [S&D Messenger: Exchanging Semantic and Domain Knowledge for Generic Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2407.07763/) (2024)
- [Exploring Token-Level Augmentation in Vision Transformer for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2503.02459/) (2025)
- [Knowledge Consultation for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2503.10693/) (2025)
- [What is the Added Value of UDA in the VFM Era?](https://mlsanity.com/paper/2504.18190/) (2025)
- [Adaptive Spatial Augmentation for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2505.23438/) (2025)
- [RS-MTDF: Multi-Teacher Distillation and Fusion for Remote Sensing Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2506.08772/) (2025)
- [Vision-Language Model Purified Semi-Supervised Semantic Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2602.00202/) (2026)
- [Bidirectional Uncertainty-Aware Region Learning for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2502.07457/) (2025)
- [BoundMatch: Boundary detection applied to semi-supervised segmentation](https://mlsanity.com/paper/2503.23519/) (2025)
- [Leveraging Out-of-Distribution Unlabeled Images: Semi-Supervised Semantic Segmentation with an Open-Vocabulary Model](https://mlsanity.com/paper/2507.03302/) (2025)
- [Style-Aware Blending and Prototype-Based Cross-Contrast Consistency for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2507.20729/) (2025)
- [Diverse Teaching and Label Propagation for Generic Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2508.08549/) (2025)
- [Harnessing Group-Oriented Consistency Constraints for Semi-Supervised Semantic Segmentation in CdZnTe Semiconductors](https://mlsanity.com/paper/2508.12766/) (2025)
- [Endo-SemiS: Towards Robust Semi-Supervised Image Segmentation for Endoscopic Video](https://mlsanity.com/paper/2512.16977/) (2025)
- [CoVar: Confidence-Variance-Guided Pseudo-Label Selection for Semi-Supervised Learning](https://mlsanity.com/paper/2601.11670/) (2026)
- [BCMDA: Bidirectional Correlation Maps Domain Adaptation for Mixed Domain Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2603.24691/) (2026)
- [Unsupervised Collaborative Domain Adaptation for Driving Scene Parsing](https://mlsanity.com/paper/2606.01818/) (2026)
- [APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms](https://mlsanity.com/paper/2606.30577/) (2026)

## Abstract (arXiv)

Semi-supervised semantic segmentation (SSSS) has been proposed to alleviate the burden of time-consuming pixel-level manual labeling, which leverages limited labeled data along with larger amounts of unlabeled data. Current state-of-the-art methods train the labeled data with ground truths and unlabeled data with pseudo labels. However, the two training flows are separate, which allows labeled data to dominate the training process, resulting in low-quality pseudo labels and, consequently, sub-optimal results. To alleviate this issue, we present AllSpark, which reborns the labeled features from unlabeled ones with the channel-wise cross-attention mechanism. We further introduce a Semantic Memory along with a Channel Semantic Grouping strategy to ensure that unlabeled features adequately represent labeled features. The AllSpark shed new light on the architecture level designs of SSSS rather than framework level, which avoids increasingly complicated training pipeline designs. It can also be regarded as a flexible bottleneck module that can be seamlessly integrated into a general transformer-based segmentation model. The proposed AllSpark outperforms existing methods across all evaluation protocols on Pascal, Cityscapes and COCO benchmarks without bells-and-whistles. Code and model weights are available at: https://github.com/xmed-lab/AllSpark.