mlsanityData to 14 Oct 2026

SIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation

arXiv 2108.12517 ↗First posted 27 Aug 2021cs.CVcited by 41

Not on the map. Why?

Abstract (arXiv)

Unlike conventional zero-shot classification, zero-shot semantic segmentation predicts a class label at the pixel level instead of the image level. When solving zero-shot semantic segmentation problems, the need for pixel-level prediction with surrounding context motivates us to incorporate spatial information using positional encoding. We improve standard positional encoding by introducing the concept of Relative Positional Encoding, which integrates spatial information at the feature level and can handle arbitrary image sizes. Furthermore, while self-training is widely used in zero-shot semantic segmentation to generate pseudo-labels, we propose a new knowledge-distillation-inspired self-training strategy, namely Annealed Self-Training, which can automatically assign different importance to pseudo-labels to improve performance. We systematically study the proposed Relative Positional Encoding and Annealed Self-Training in a comprehensive experimental evaluation, and our empirical results confirm the effectiveness of our method on three benchmark datasets.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Builds on 40 of its 64 references that are in the map. First posted on arXiv 27 Aug 2021.

Builds on

40 of 64 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2013Auto-Encoding Variational Bayescited by 4,053
  11. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  12. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
28 more
  1. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  2. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  3. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  4. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  5. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  6. 2015Generative Moment Matching Networkscited by 257
  7. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  8. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 255
  9. 2019Zero-Shot Semantic Segmentationcited by 245
  10. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  11. 2018Zero-Shot Object Detectioncited by 189
  12. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  13. 2015Label-Embedding for Image Classificationcited by 165
  14. 2018Zero-shot Recognition via Semantic Embeddings and Knowledge Graphscited by 155
  15. 2013Zero-Shot Learning by Convex Combination of Semantic Embeddingscited by 135
  16. 2016Latent Embeddings for Zero-shot Classificationcited by 129
  17. 2020Context-aware Feature Generation for Zero-shot Semantic Segmentationcited by 94
  18. 2018Rethinking Knowledge Graph Propagation for Zero-Shot Learningcited by 87
  19. 2018Multi-modal Cycle-consistent Generalized Zero-Shot Learningcited by 87
  20. 2015Zero-Shot Learning via Semantic Similarity Embeddingcited by 74
  21. 2020Improve Transformer Models with Better Relative Position Embeddingscited by 73
  22. 2017Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networkscited by 66
  23. 2017A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoderscited by 64
  24. 2018Zero-Shot Object Detection: Learning to Simultaneously Recognize and Localize Novel Conceptscited by 48
  25. 2018Zero-Shot Object Detection by Hybrid Region Embeddingcited by 32
  26. 2017Semantically Consistent Regularization for Zero-Shot Recognitioncited by 14
  27. 2018Polarity Loss for Zero-shot Object Detectioncited by 14
  28. 2016How to Transfer? Zero-Shot Object Recognition via Hierarchical Transfer of Semantic Attributescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 3; 2023: 19; 2024: 6; 2025: 9; 2026: 1.

41 papers.

Sort:

By area:

  1. 2021Decoupling Zero-Shot Semantic Segmentationcited by 199
  2. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  3. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  4. 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Modelscited by 162
  5. 2023DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Modelscited by 123
  6. 2023Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervisioncited by 99
  7. 2021Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labelingcited by 84
  8. 2023Open-vocabulary Object Segmentation with Diffusion Modelscited by 82
  9. 2022Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embeddingcited by 67
  10. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 64
  11. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  12. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 61
29 more
  1. 2023Primitive Generation and Semantic-related Alignment for Universal Zero-Shot Segmentationcited by 28
  2. 2023See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Datacited by 21
  3. 2024Cascade-CLIP: Cascaded Vision-Language Embeddings Alignment for Zero-Shot Semantic Segmentationcited by 17
  4. 2023Semantic-Promoted Debiasing and Background Disambiguation for Zero-Shot Instance Segmentationcited by 16
  5. 2023IFSeg: Image-free Semantic Segmentation via Vision-Language Modelcited by 15
  6. 2023Towards Training-free Open-world Segmentation via Image Prompt Foundation Modelscited by 15
  7. 2023TagCLIP: Improving Discrimination Ability of Open-Vocabulary Semantic Segmentationcited by 14
  8. 2024IPO: Interpretable Prompt Optimization for Vision-Language Modelscited by 14
  9. 2023Spectral Prompt Tuning:Unveiling Unseen Classes for Zero-Shot Semantic Segmentationcited by 11
  10. 2023MVP-SEG: Multi-View Prompt Learning for Open-Vocabulary Semantic Segmentationcited by 10
  11. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
  12. 2023CLIP Is Also a Good Teacher: A New Learning Framework for Inductive Zero-shot Semantic Segmentationcited by 9
  13. 2024Language-Driven Visual Consensus for Zero-Shot Semantic Segmentationcited by 6
  14. 2023Instance-level Few-shot Learning with Class Hierarchy Miningcited by 5
  15. 2024AlignZeg: Mitigating Objective Misalignment for Zero-shot Semantic Segmentationcited by 5
  16. 2025SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Modelscited by 5
  17. 2025Target Refocusing via Attention Redistribution for Open-Vocabulary Semantic Segmentation: An Explainability Perspectivecited by 5
  18. 2023A Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic Segmentationcited by 3
  19. 2024Free-Mask: A Novel Paradigm of Integration Between the Segmentation Diffusion Model and Image Editingcited by 2
  20. 2025Split Matching for Inductive Zero-shot Semantic Segmentationcited by 2
  21. 2025Novel Category Discovery with X-Agent Attention for Open-Vocabulary Semantic Segmentationcited by 2
  22. 2021SoK: Vehicle Orientation Representations for Deep Rotation Estimationcited by 1
  23. 2024Generalizable Semantic Vision Query Generation for Zero-shot Panoptic and Semantic Segmentationcited by 1
  24. 2025SynthSeg-Agents: Multi-Agent Synthetic Data Generation for Zero-Shot Weakly Supervised Semantic Segmentationcited by 1
  25. 2023Class Enhancement Losses with Pseudo Labels for Zero-shot Semantic Segmentationcited by 0
  26. 2025Semantic Segmentation of Transparent and Opaque Drinking Glasses with the Help of Zero-shot Learningcited by 0
  27. 2025Bridge the Gap Between Visual and Linguistic Comprehension for Generalized Zero-shot Semantic Segmentationcited by 0
  28. 2025Partial CLIP is Enough: Chimera-Seg for Zero-shot Semantic Segmentationcited by 0
  29. 2026Direct Segmentation without Logits Optimization for Training-Free Open-Vocabulary Semantic Segmentationcited by 0