# In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation

arXiv: https://arxiv.org/abs/2408.04961 · HTML: https://mlsanity.com/paper/2408.04961/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 90 of its 125 references that are in the map.
- First posted on arXiv 9 Aug 2024.

## Builds on (most cited first, 30 of 90)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)

## Built on by (most cited first, 30 of 38)

- [DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment](https://mlsanity.com/paper/2412.16334/) (2024)
- [Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2411.19331/) (2024)
- [Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentation](https://mlsanity.com/paper/2411.09219/) (2024)
- [CorrCLIP: Reconstructing Patch Correlations in CLIP for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2411.10086/) (2024)
- [Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentation](https://mlsanity.com/paper/2411.15869/) (2024)
- [ResCLIP: Residual Attention for Training-free Dense Vision-language Inference](https://mlsanity.com/paper/2411.15851/) (2024)
- [SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2410.01768/) (2024)
- [Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2411.17150/) (2024)
- [Mosaic3D: Foundation Dataset and Model for Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2502.02548/) (2025)
- [LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation](https://mlsanity.com/paper/2503.19777/) (2025)
- [Exploring the Underwater World Segmentation without Extra Training](https://mlsanity.com/paper/2511.07923/) (2025)
- [SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images](https://mlsanity.com/paper/2512.08730/) (2025)
- [Training-Free Class Purification for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2508.00557/) (2025)
- [Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images](https://mlsanity.com/paper/2508.18067/) (2025)
- [TextRegion: Text-Aligned Region Tokens from Frozen Image-Text Models](https://mlsanity.com/paper/2505.23769/) (2025)
- [Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scale](https://mlsanity.com/paper/2506.12009/) (2025)
- [Target Refocusing via Attention Redistribution for Open-Vocabulary Semantic Segmentation: An Explainability Perspective](https://mlsanity.com/paper/2511.16170/) (2025)
- [Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2602.06333/) (2026)
- [Open-Vocabulary Domain Generalization in Urban-Scene Segmentation](https://mlsanity.com/paper/2602.18853/) (2026)
- [Unsupervised Segmentation by Diffusing, Walking and Cutting](https://mlsanity.com/paper/2412.04678/) (2024)
- [PEARL: Geometry Aligns Semantics for Training-Free Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2603.21528/) (2026)
- [Enabling Training-Free Text-Based Remote Sensing Segmentation](https://mlsanity.com/paper/2602.17799/) (2026)
- [Looking Beyond the Window: Global-Local Aligned CLIP for Training-free Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2603.23030/) (2026)
- [VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inference](https://mlsanity.com/paper/2605.12325/) (2026)
- [Improving Visual Discriminability of CLIP for Training-Free Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2510.23894/) (2025)
- [DINO Soars: DINOv3 for Open-Vocabulary Semantic Segmentation of Remote Sensing Imagery](https://mlsanity.com/paper/2605.03175/) (2026)
- [DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2607.21371/) (2026)
- [Memory-Modular Classification: Learning to Generalize with Memory Replacement](https://mlsanity.com/paper/2504.06021/) (2025)
- [A Survey on Training-free Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2505.22209/) (2025)
- [Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders](https://mlsanity.com/paper/2509.00177/) (2025)

## Abstract (arXiv)

We present lazy visual grounding, a two-stage approach of unsupervised object mask discovery followed by object grounding, for open-vocabulary semantic segmentation. Plenty of the previous art casts this task as pixel-to-text classification without object-level comprehension, leveraging the image-to-text classification capability of pretrained vision-and-language models. We argue that visual objects are distinguishable without the prior text information as segmentation is essentially a vision task. Lazy visual grounding first discovers object masks covering an image with iterative Normalized cuts and then later assigns text on the discovered objects in a late interaction manner. Our model requires no additional training yet shows great performance on five public datasets: Pascal VOC, Pascal Context, COCO-object, COCO-stuff, and ADE 20K. Especially, the visually appealing segmentation results demonstrate the model capability to localize objects precisely. Paper homepage: https://cvlab.postech.ac.kr/research/lazygrounding