# Learning Visual Proxy for Compositional Zero-Shot Learning

arXiv: https://arxiv.org/abs/2501.13859 · HTML: https://mlsanity.com/paper/2501.13859/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 23 Jan 2025.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [Task-Driven Modular Networks for Zero-Shot Compositional Learning](https://mlsanity.com/paper/1905.05908/) (2019)
- [Learning Graph Embeddings for Compositional Zero-shot Learning](https://mlsanity.com/paper/2102.01987/) (2021)
- [Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2101.12609/) (2021)
- [Symmetry and Group in Attribute-Object Compositions](https://mlsanity.com/paper/2004.00587/) (2020)
- [A causal view of compositional zero-shot recognition](https://mlsanity.com/paper/2006.14610/) (2020)
- [Learning to Compose Soft Prompts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.03574/) (2022)
- [Disentangling Visual Embeddings for Attributes and Objects](https://mlsanity.com/paper/2205.08536/) (2022)
- [Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2206.14475/) (2022)
- [Independent Prototype Propagation for Zero-Shot Compositionality](https://mlsanity.com/paper/2106.00305/) (2021)
- [KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2205.06784/) (2022)
- [Learning Conditional Attributes for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.17940/) (2023)
- [Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2303.15230/) (2023)
- [Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP](https://mlsanity.com/paper/2310.19752/) (2023)
- [Prompting Language-Informed Distribution for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.14428/) (2023)
- [Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning](https://mlsanity.com/paper/2402.17251/) (2024)
- [DRPT: Disentangled and Recurrent Prompt Tuning for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.01239/) (2023)
- [Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2308.04016/) (2023)
- [GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2311.05729/) (2023)
- [On Leveraging Variational Graph Embeddings for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.11848/) (2022)
- [Prompting Large Pre-trained Vision-Language Models For Compositional Concept Learning](https://mlsanity.com/paper/2211.05077/) (2022)
- [Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning](https://mlsanity.com/paper/2312.15923/) (2023)

## Built on by (most cited first, 2 of 2)

- [FlowComposer: Composable Flows for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2603.16641/) (2026)
- [From Model Patterns to Abstract Semantics in Compositional Zero-Shot Learning](https://mlsanity.com/paper/2609.15649/) (2026)

## Abstract (arXiv)

Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions by leveraging knowledge from seen compositions. Current methods align textual prototypes with visual features via Vision-Language Models (VLMs), but suffer from two limitations: (1) modality gaps hinder the discrimination of semantically similar pairs, and (2) single-modal textual prototypes lack fine-grained visual cues. In this paper, we introduce Visual Proxy Learning, a method that reduces modality gaps and enhances compositional generalization. We initialize visual proxies for attributes, objects, and their compositions using text representations and optimize the visual space to capture fine-grained cues, improving visual representations. Additionally, we propose Cross-Modal Joint Learning (CMJL), which imposes cross-modal constraints between the text-image and fine-grained visual spaces, improving generalization for unseen compositions and discriminating similar pairs. Experiments show state-of-the-art performance in closed-world scenarios and competitive results in open-world settings across four CZSL benchmarks, demonstrating the effectiveness of our approach in compositional generalization.