# LCCo: Lending CLIP to Co-Segmentation

arXiv: https://arxiv.org/abs/2308.11506 · HTML: https://mlsanity.com/paper/2308.11506/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Extract Free Dense Labels from CLIP](https://mlsanity.com/paper/2112.01071/) (2021)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [Decoupling Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2112.07910/) (2021)
- [ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2212.03588/) (2022)
- [A Unified Transformer Framework for Group-based Segmentation: Co-Segmentation, Co-Saliency Detection and Video Salient Object Detection](https://mlsanity.com/paper/2203.04708/) (2022)
- [ExFuse: Enhancing Feature Fusion for Semantic Segmentation](https://mlsanity.com/paper/1804.03821/) (2018)
- [Deep Object Co-Segmentation](https://mlsanity.com/paper/1804.06423/) (2018)
- [Edit Everything: A Text-Guided Generative System for Images Editing](https://mlsanity.com/paper/2304.14006/) (2023)
- [CycleSegNet: Object Co-segmentation with Cycle Refinement and Region Correspondence](https://mlsanity.com/paper/2101.01308/) (2021)
- [Semantic Aware Attention Based Deep Object Co-segmentation](https://mlsanity.com/paper/1810.06859/) (2018)
- [Deep Object Co-segmentation via Spatial-Semantic Network Modulation](https://mlsanity.com/paper/1911.12950/) (2019)
- [Learning Co-segmentation by Segment Swapping for Retrieval and Discovery](https://mlsanity.com/paper/2110.15904/) (2021)

## Built on by (most cited first, 1 of 1)

- [CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Models](https://mlsanity.com/paper/2412.19331/) (2024)

## Abstract (arXiv)

This paper studies co-segmenting the common semantic object in a set of images. Existing works either rely on carefully engineered networks to mine the implicit semantic information in visual features or require extra data (i.e., classification labels) for training. In this paper, we leverage the contrastive language-image pre-training framework (CLIP) for the task. With a backbone segmentation network that independently processes each image from the set, we introduce semantics from CLIP into the backbone features, refining them in a coarse-to-fine manner with three key modules: i) an image set feature correspondence module, encoding global consistent semantic information of the image set; ii) a CLIP interaction module, using CLIP-mined common semantics of the image set to refine the backbone feature; iii) a CLIP regularization module, drawing CLIP towards this co-segmentation task, identifying the best CLIP semantic and using it to regularize the backbone feature. Experiments on four standard co-segmentation benchmark datasets show that the performance of our method outperforms state-of-the-art methods.