# Co-Segmentation without any Pixel-level Supervision with Application to Large-Scale Sketch Classification

arXiv: https://arxiv.org/abs/2410.13582 · HTML: https://mlsanity.com/paper/2410.13582/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 58 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [SelfReg: Self-supervised Contrastive Regularization for Domain Generalization](https://mlsanity.com/paper/2104.09841/) (2021)
- [Learning to Diversify for Single Domain Generalization](https://mlsanity.com/paper/2108.11726/) (2021)
- [Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut](https://mlsanity.com/paper/2202.11539/) (2022)
- [Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization](https://mlsanity.com/paper/2205.07839/) (2022)
- [ReCo: Retrieve and Co-segment for Zero-shot Transfer](https://mlsanity.com/paper/2206.07045/) (2022)
- [Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval](https://mlsanity.com/paper/1904.03451/) (2019)
- [Bi-Directional Cascade Network for Perceptual Edge Detection](https://mlsanity.com/paper/1902.10903/) (2019)
- [Unsupervised Salient Object Detection with Spectral Cluster Voting](https://mlsanity.com/paper/2203.12614/) (2022)
- [A Unified Transformer Framework for Group-based Segmentation: Co-Segmentation, Co-Saliency Detection and Video Salient Object Detection](https://mlsanity.com/paper/2203.04708/) (2022)
- [Gradient-Induced Co-Saliency Detection](https://mlsanity.com/paper/2004.13364/) (2020)
- [Deep Object Co-Segmentation](https://mlsanity.com/paper/1804.06423/) (2018)
- [Group Collaborative Learning for Co-Salient Object Detection](https://mlsanity.com/paper/2104.01108/) (2021)
- [Summarize and Search: Learning Consensus-aware Dynamic Convolution for Co-Saliency Detection](https://mlsanity.com/paper/2110.00338/) (2021)
- [CycleSegNet: Object Co-segmentation with Cycle Refinement and Region Correspondence](https://mlsanity.com/paper/2101.01308/) (2021)
- [Democracy Does Matter: Comprehensive Feature Mining for Co-Salient Object Detection](https://mlsanity.com/paper/2203.05787/) (2022)
- [Deep Object Co-segmentation via Spatial-Semantic Network Modulation](https://mlsanity.com/paper/1911.12950/) (2019)
- [Asymmetric Feature Maps with Application to Sketch Based Retrieval](https://mlsanity.com/paper/1704.03946/) (2017)
- [Generalisation and Sharing in Triplet Convnets for Sketch based Visual Search](https://mlsanity.com/paper/1611.05301/) (2016)
- [Edge Augmentation for Large-Scale Sketch Recognition without Sketches](https://mlsanity.com/paper/2202.13164/) (2022)

## Built on by (most cited first, 1 of 1)

- [LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation](https://mlsanity.com/paper/2503.19777/) (2025)

## Abstract (arXiv)

This work proposes a novel method for object co-segmentation, i.e. pixel-level localization of a common object in a set of images, that uses no pixel-level supervision for training. Two pre-trained Vision Transformer (ViT) models are exploited: ImageNet classification-trained ViT, whose features are used to estimate rough object localization through intra-class token relevance, and a self-supervised DINO-ViT for intra-image token relevance. On recent challenging benchmarks, the method achieves state-of-the-art performance among methods trained with the same level of supervision (image labels) while being competitive with methods trained with pixel-level supervision (binary masks). The benefits of the proposed co-segmentation method are further demonstrated in the task of large-scale sketch recognition, that is, the classification of sketches into a wide range of categories. The limited amount of hand-drawn sketch training data is leveraged by exploiting readily available image-level-annotated datasets of natural images containing a large number of classes. To bridge the domain gap, the classifier is trained on a sketch-like proxy domain derived from edges detected on natural images. We show that sketch recognition significantly benefits when the classifier is trained on sketch-like structures extracted from the co-segmented area rather than from the full image. Code: https://github.com/nikosips/CBNC .