mlsanityData to 7 Oct 2026

Localizing Objects with Self-Supervised Transformers and no Labels

arXiv 2109.14279 ↗First posted 29 Sep 2021cs.CVcited by 227

Not on the map. Why?

Abstract (arXiv)

Localizing objects in image collections without supervision can help to avoid expensive annotation campaigns. We propose a simple approach to this problem, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner. Our method, LOST, does not require any external object proposal nor any exploration of the image collection; it operates on a single image. Yet, we outperform state-of-the-art object discovery methods by up to 8 CorLoc points on PASCAL VOC 2012. We also show that training a class-agnostic detector on the discovered objects boosts results by another 7 points. Moreover, we show promising results on the unsupervised object discovery task. The code to reproduce our results can be found at https://github.com/valeoai/LOST.

Key facts. Cited by 227 papers in the map (data through 7 Oct 2026). Builds on 66 of its 82 references that are in the map. First posted on arXiv 29 Sep 2021.

Builds on

66 of 82 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2017Mask R-CNNcited by 7,259
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  12. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
38 more
  1. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2015Fast R-CNNcited by 2,930
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  8. 2017Non-local Neural Networkscited by 2,290
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  11. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  12. 2016Colorful Image Colorizationcited by 1,113
  13. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  14. 2021Scaling Vision Transformerscited by 993
  15. 2020Object-Centric Learning with Slot Attentioncited by 865
  16. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  17. 2021Going deeper with Image Transformerscited by 759
  18. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  19. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  20. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  21. 2021Intriguing Properties of Vision Transformerscited by 446
  22. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  23. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  24. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  25. 2017Relation Networks for Object Detectioncited by 305
  26. 2021Unbiased Teacher for Semi-Supervised Object Detectioncited by 305
  27. 2019On the Relationship between Self-Attention and Convolutional Layerscited by 299
  28. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  29. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  30. 2020Self-Supervised Learning of Audio-Visual Objects from Videocited by 186
  31. 2015Weakly Supervised Deep Detection Networkscited by 183
  32. 2014Object Detectors Emerge in Deep Scene CNNscited by 178
  33. 2017Data Distillation: Towards Omni-Supervised Learningcited by 126
  34. 2021Self-Supervised Learning with Swin Transformerscited by 121
  35. 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
  36. 2018PCL: Proposal Cluster Learning for Weakly Supervised Object Detectioncited by 97
  37. 2021Training Vision Transformers for Image Retrievalcited by 90
  38. 2020Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detectioncited by 80

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 36; 2023: 57; 2024: 52; 2025: 44; 2026: 35.

227 papers.

Sort:

By area:

  1. 2023Vision Transformers Need Registerscited by 844
  2. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
  3. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  4. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  5. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  6. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 186
  7. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 166
  8. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  9. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  10. 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 115
  11. 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 111
  12. 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 110
38 more
  1. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  2. 2022FreeSOLO: Learning to Segment Objects without Annotationscited by 102
  3. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  4. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 94
  5. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  6. 2021Class-agnostic Object Detection with Multi-modal Transformercited by 85
  7. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 78
  8. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  9. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  10. 2023LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affordance Groundingcited by 64
  11. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 63
  12. 2022Unsupervised Salient Object Detection with Spectral Cluster Votingcited by 56
  13. 2022Discovering Object Masks with Transformers for Unsupervised Semantic Segmentationcited by 54
  14. 2022Unsupervised Semantic Segmentation with Self-supervised Object-centric Representationscited by 54
  15. 2023DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discoverycited by 53
  16. 2022Self-Supervised Vision Transformers Learn Visual Concepts in Histopathologycited by 49
  17. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  18. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  19. 2022Teaching Matters: Investigating the Role of Supervision in Vision Transformerscited by 45
  20. 2023Towards In-context Scene Understandingcited by 45
  21. 2023Three Pillars improving Vision Foundation Model Distillation for Lidarcited by 43
  22. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  23. 2022ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentationcited by 42
  24. 2024Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentationcited by 42
  25. 2023Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled videocited by 40
  26. 2024Denoising Vision Transformerscited by 40
  27. 2023CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Freecited by 39
  28. 2025Vision Transformers Don't Need Trained Registerscited by 39
  29. 2024In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentationcited by 38
  30. 2022Self-Guided Diffusion Modelscited by 36
  31. 2023VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentationcited by 36
  32. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  33. 2023Unsupervised Universal Image Segmentationcited by 34
  34. 2024Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentationcited by 34
  35. 2024CorrCLIP: Reconstructing Patch Correlations in CLIP for Open-Vocabulary Semantic Segmentationcited by 33
  36. 2023Text-Driven Image Editing via Learnable Regionscited by 31
  37. 2023Neural Congealing: Aligning Images to a Joint Semantic Atlascited by 30
  38. 2024Segment Anything without Supervisioncited by 27

50 most cited of 227 All 227 (JSON)