# TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cut

arXiv: https://arxiv.org/abs/2209.00383 · HTML: https://mlsanity.com/paper/2209.00383/

## Key facts

- Cited by 113 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 87 references that are in the map.
- First posted on arXiv 1 Sep 2022.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization](https://mlsanity.com/paper/2205.07839/) (2022)
- [Self-supervised Video Object Segmentation by Motion Grouping](https://mlsanity.com/paper/2104.07658/) (2021)
- [Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2003.13045/) (2020)
- [MST: Masked Self-Supervised Transformer for Visual Representation](https://mlsanity.com/paper/2106.05656/) (2021)
- [ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation](https://mlsanity.com/paper/1911.11789/) (2019)
- [Salient Object Detection: A Discriminative Regional Feature Integration Approach](https://mlsanity.com/paper/1410.5926/) (2014)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detection](https://mlsanity.com/paper/2004.04725/) (2020)
- [Unsupervised Moving Object Detection via Contextual Information Separation](https://mlsanity.com/paper/1901.03360/) (2019)
- [Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposals](https://mlsanity.com/paper/1501.06170/) (2015)

## Built on by (most cited first, 30 of 113)

- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [Cut and Learn for Unsupervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2301.11320/) (2023)
- [Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learning](https://mlsanity.com/paper/2212.04994/) (2022)
- [Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2411.19331/) (2024)
- [VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation](https://mlsanity.com/paper/2308.14710/) (2023)
- [Unsupervised Universal Image Segmentation](https://mlsanity.com/paper/2312.17243/) (2023)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [Adaptive Slot Attention: Object Discovery with Dynamic Slot Number](https://mlsanity.com/paper/2406.09196/) (2024)
- [Segment Anything without Supervision](https://mlsanity.com/paper/2406.20081/) (2024)
- [LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptors](https://mlsanity.com/paper/2403.14625/) (2024)
- [DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cut](https://mlsanity.com/paper/2406.02842/) (2024)
- [Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers?](https://mlsanity.com/paper/2510.24709/) (2025)
- [Few-Shot Panoptic Segmentation With Foundation Models](https://mlsanity.com/paper/2309.10726/) (2023)
- [CuVLER: Enhanced Unsupervised Object Discoveries through Exhaustive Self-Supervised Transformers](https://mlsanity.com/paper/2403.07700/) (2024)
- [Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Grouping](https://mlsanity.com/paper/2304.08025/) (2023)
- [Weakly-supervised Contrastive Learning for Unsupervised Object Discovery](https://mlsanity.com/paper/2307.03376/) (2023)
- [SANGRIA: Surgical Video Scene Graph Optimization for Surgical Workflow Prediction](https://mlsanity.com/paper/2407.20214/) (2024)
- [Understanding Masked Autoencoders From a Local Contrastive Perspective](https://mlsanity.com/paper/2310.01994/) (2023)
- [Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation](https://mlsanity.com/paper/2311.17893/) (2023)
- [HASSOD: Hierarchical Adaptive Self-Supervised Object Detection](https://mlsanity.com/paper/2402.03311/) (2024)
- [Scene-Centric Unsupervised Panoptic Segmentation](https://mlsanity.com/paper/2504.01955/) (2025)
- [Sub-token ViT Embedding via Stochastic Resonance Transformers](https://mlsanity.com/paper/2310.03967/) (2023)
- [Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations](https://mlsanity.com/paper/2312.06716/) (2023)
- [Boosting Unsupervised Semantic Segmentation with Principal Mask Proposals](https://mlsanity.com/paper/2404.16818/) (2024)
- [Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentation](https://mlsanity.com/paper/2411.10411/) (2024)
- [Flow-Anything: Learning Real-World Optical Flow Estimation from Large-Scale Single-view Images](https://mlsanity.com/paper/2506.07740/) (2025)
- [LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-training](https://mlsanity.com/paper/2308.11239/) (2023)
- [Cutting the Skip: Training Residual-Free Transformers](https://mlsanity.com/paper/2510.00345/) (2025)
- [HiERO: understanding the hierarchy of human behavior enhances reasoning on egocentric videos](https://mlsanity.com/paper/2505.12911/) (2025)
- [FreePoint: Unsupervised Point Cloud Instance Segmentation](https://mlsanity.com/paper/2305.06973/) (2023)

## Abstract (arXiv)

In this paper, we describe a graph-based algorithm that uses the features obtained by a self-supervised transformer to detect and segment salient objects in images and videos. With this approach, the image patches that compose an image or video are organised into a fully connected graph, where the edge between each pair of patches is labeled with a similarity score between patches using features learned by the transformer. Detection and segmentation of salient objects is then formulated as a graph-cut problem and solved using the classical Normalized Cut algorithm. Despite the simplicity of this approach, it achieves state-of-the-art results on several common image and video detection and segmentation tasks. For unsupervised object discovery, this approach outperforms the competing approaches by a margin of 6.1%, 5.7%, and 2.6%, respectively, when tested with the VOC07, VOC12, and COCO20K datasets. For the unsupervised saliency detection task in images, this method improves the score for Intersection over Union (IoU) by 4.4%, 5.6% and 5.2%. When tested with the ECSSD, DUTS, and DUT-OMRON datasets, respectively, compared to current state-of-the-art techniques. This method also achieves competitive results for unsupervised video object segmentation tasks with the DAVIS, SegTV2, and FBMS datasets.