# From Pixel to Patch: Synthesize Context-aware Features for Zero-shot Semantic Segmentation

arXiv: https://arxiv.org/abs/2009.12232 · HTML: https://mlsanity.com/paper/2009.12232/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2020.

## Built on by (most cited first, 11 of 11)

- [A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model](https://mlsanity.com/paper/2112.14757/) (2021)
- [Decoupling Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2112.07910/) (2021)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Perceptual Grouping in Contrastive Vision-Language Models](https://mlsanity.com/paper/2210.09996/) (2022)
- [Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models](https://mlsanity.com/paper/2210.15138/) (2022)
- [Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview](https://mlsanity.com/paper/2211.08352/) (2022)
- [Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2211.00849/) (2022)
- [Deep Rank-Consistent Pyramid Model for Enhanced Crowd Counting](https://mlsanity.com/paper/2201.04819/) (2022)
- [STARS: Zero-shot Sim-to-Real Transfer for Segmentation of Shipwrecks in Sonar Imagery](https://mlsanity.com/paper/2310.01667/) (2023)
- [MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentation](https://mlsanity.com/paper/2401.11738/) (2024)
- [Semantic Segmentation of Transparent and Opaque Drinking Glasses with the Help of Zero-shot Learning](https://mlsanity.com/paper/2503.15004/) (2025)

## Abstract (arXiv)

Zero-shot learning has been actively studied for image classification task to relieve the burden of annotating image labels. Interestingly, semantic segmentation task requires more labor-intensive pixel-wise annotation, but zero-shot semantic segmentation has only attracted limited research interest. Thus, we focus on zero-shot semantic segmentation, which aims to segment unseen objects with only category-level semantic representations provided for unseen categories. In this paper, we propose a novel Context-aware feature Generation Network (CaGNet), which can synthesize context-aware pixel-wise visual features for unseen categories based on category-level semantic representations and pixel-wise contextual information. The synthesized features are used to finetune the classifier to enable segmenting unseen objects. Furthermore, we extend pixel-wise feature generation and finetuning to patch-wise feature generation and finetuning, which additionally considers inter-pixel relationship. Experimental results on Pascal-VOC, Pascal-Context, and COCO-stuff show that our method significantly outperforms the existing zero-shot semantic segmentation methods. Code is available at https://github.com/bcmi/CaGNetv2-Zero-Shot-Semantic-Segmentation.