# MSCI: Addressing CLIP's Inherent Limitations for Compositional Zero-Shot Learning

arXiv: https://arxiv.org/abs/2505.10289 · HTML: https://mlsanity.com/paper/2505.10289/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Task-Driven Modular Networks for Zero-Shot Compositional Learning](https://mlsanity.com/paper/1905.05908/) (2019)
- [Learning Graph Embeddings for Compositional Zero-shot Learning](https://mlsanity.com/paper/2102.01987/) (2021)
- [TransVPR: Transformer-based place recognition with multi-level attention aggregation](https://mlsanity.com/paper/2201.02001/) (2022)
- [Learning to Compose Soft Prompts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.03574/) (2022)
- [Disentangling Visual Embeddings for Attributes and Objects](https://mlsanity.com/paper/2205.08536/) (2022)
- [Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2206.14475/) (2022)
- [KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2205.06784/) (2022)
- [Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2211.10681/) (2022)
- [Learning Attention as Disentangler for Compositional Zero-shot Learning](https://mlsanity.com/paper/2303.15111/) (2023)
- [Learning Conditional Attributes for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.17940/) (2023)
- [Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2303.15230/) (2023)
- [Prompting Language-Informed Distribution for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.14428/) (2023)
- [DuAT: Dual-Aggregation Transformer Network for Medical Image Segmentation](https://mlsanity.com/paper/2212.11677/) (2022)
- [Learning Attention Propagation for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2210.11557/) (2022)
- [Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning](https://mlsanity.com/paper/2402.17251/) (2024)
- [GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2311.05729/) (2023)
- [Distilled Reverse Attention Network for Open-world Compositional Zero-Shot Learning](https://mlsanity.com/paper/2303.00404/) (2023)
- [Attention Is Not the Only Choice: Counterfactual Reasoning for Path-Based Explainable Recommendation](https://mlsanity.com/paper/2401.05744/) (2024)
- [PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection](https://mlsanity.com/paper/2406.02318/) (2024)
- [Toward Structure Fairness in Dynamic Graph Embedding: A Trend-aware Dual Debiasing Approach](https://mlsanity.com/paper/2406.13201/) (2024)
- [TransURL: Improving malicious URL detection with multi-layer Transformer encoding and multi-scale pyramid features](https://mlsanity.com/paper/2312.00508/) (2023)

## Built on by (most cited first, 1 of 1)

- [Unified Multi-Dataset Training for TBPS](https://mlsanity.com/paper/2601.14978/) (2026)

## Abstract (arXiv)

Compositional Zero-Shot Learning (CZSL) aims to recognize unseen state-object combinations by leveraging known combinations. Existing studies basically rely on the cross-modal alignment capabilities of CLIP but tend to overlook its limitations in capturing fine-grained local features, which arise from its architectural and training paradigm. To address this issue, we propose a Multi-Stage Cross-modal Interaction (MSCI) model that effectively explores and utilizes intermediate-layer information from CLIP's visual encoder. Specifically, we design two self-adaptive aggregators to extract local information from low-level visual features and integrate global information from high-level visual features, respectively. These key information are progressively incorporated into textual representations through a stage-by-stage interaction mechanism, significantly enhancing the model's perception capability for fine-grained local visual information. Additionally, MSCI dynamically adjusts the attention weights between global and local visual information based on different combinations, as well as different elements within the same combination, allowing it to flexibly adapt to diverse scenarios. Experiments on three widely used datasets fully validate the effectiveness and superiority of the proposed model. Data and code are available at https://github.com/ltpwy/MSCI.