# Exploring Token-Level Augmentation in Vision Transformer for Semi-Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/2503.02459 · HTML: https://mlsanity.com/paper/2503.02459/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning](https://mlsanity.com/paper/2007.07936/) (2020)
- [ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.05095/) (2021)
- [Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2208.09910/) (2022)
- [A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation](https://mlsanity.com/paper/2104.07256/) (2021)
- [Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation](https://mlsanity.com/paper/2107.11279/) (2021)
- [Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2212.04976/) (2022)
- [Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation](https://mlsanity.com/paper/2308.01045/) (2023)
- [TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers](https://mlsanity.com/paper/2207.08409/) (2022)
- [CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2306.04300/) (2023)
- [Conflict-Based Cross-View Consistency for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2303.01276/) (2023)
- [AllSpark: Reborn Labeled Features from Unlabeled in Transformer for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2403.01818/) (2024)
- [Unbiased Subclass Regularization for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.10026/) (2022)
- [Switching Temporary Teachers for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2310.18640/) (2023)
- [PrevMatch: Revisiting and Maximizing Temporal Knowledge in Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2405.20610/) (2024)
- [Uncertainty-Participation Context Consistency Learning for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2412.17331/) (2024)

## Built on by (most cited first, 1 of 1)

- [ModuSeg: Decoupling Object Discovery and Semantic Retrieval for Training-Free Weakly Supervised Segmentation](https://mlsanity.com/paper/2604.07021/) (2026)

## Abstract (arXiv)

Semi-supervised semantic segmentation has witnessed remarkable advancements in recent years. However, existing algorithms are based on convolutional neural networks and directly applying them to Vision Transformers poses certain limitations due to conceptual disparities. To this end, we propose TokenMix, a data augmentation technique specifically designed for semi-supervised semantic segmentation with Vision Transformers. TokenMix aligns well with the global attention mechanism by mixing images at the token level, enhancing learning capability for contextual information among image patches. We further incorporate image augmentation and feature augmentation to promote the diversity of augmentation. Moreover, to enhance consistency regularization, we propose a dual-branch framework where each branch applies image and feature augmentation to the input image. We conduct extensive experiments across multiple benchmark datasets, including Pascal VOC 2012, Cityscapes, and COCO. Results suggest that the proposed method outperforms state-of-the-art algorithms with notably observed accuracy improvement, especially under limited fine annotations.