# Disentangling Visual Embeddings for Attributes and Objects

arXiv: https://arxiv.org/abs/2205.08536 · HTML: https://mlsanity.com/paper/2205.08536/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 59 references that are in the map.
- First posted on arXiv 17 May 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Learning to Detect Human-Object Interactions](https://mlsanity.com/paper/1702.05448/) (2017)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Task-Driven Modular Networks for Zero-Shot Compositional Learning](https://mlsanity.com/paper/1905.05908/) (2019)
- [IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval](https://mlsanity.com/paper/2003.03772/) (2020)
- [Learning Graph Embeddings for Compositional Zero-shot Learning](https://mlsanity.com/paper/2102.01987/) (2021)
- [Learning Human-Object Interaction Detection using Interaction Points](https://mlsanity.com/paper/2003.14023/) (2020)
- [Learning to Predict Visual Attributes in the Wild](https://mlsanity.com/paper/2106.09707/) (2021)
- [Symmetry and Group in Attribute-Object Compositions](https://mlsanity.com/paper/2004.00587/) (2020)
- [A causal view of compositional zero-shot recognition](https://mlsanity.com/paper/2006.14610/) (2020)
- [Learning Graph Embeddings for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2105.01017/) (2021)
- [Joint Discovery of Object States and Manipulation Actions](https://mlsanity.com/paper/1702.02738/) (2017)
- [Attributes as Operators: Factorizing Unseen Attribute-Object Compositions](https://mlsanity.com/paper/1803.09851/) (2018)
- [Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition](https://mlsanity.com/paper/2108.04603/) (2021)
- [Zero-Shot Compositional Concept Learning](https://mlsanity.com/paper/2107.05176/) (2021)
- [COBE: Contextualized Object Embeddings from Narrated Instructional Video](https://mlsanity.com/paper/2007.07306/) (2020)
- [Recognizing Manipulation Actions from State-Transformations](https://mlsanity.com/paper/1906.05147/) (2019)
- [Learning to Infer Unseen Single-/Multi-Attribute-Object Compositions with Graph Networks](https://mlsanity.com/paper/2010.14343/) (2020)
- [The 'Power of Then': The Uniquely Human Capacity to Imagine Beyond the Present](https://mlsanity.com/paper/1501.04383/) (2015)

## Built on by (most cited first, 30 of 55)

- [OneRestore: A Universal Restoration Framework for Composite Degradation](https://mlsanity.com/paper/2407.04621/) (2024)
- [OvarNet: Towards Open-vocabulary Object Attribute Recognition](https://mlsanity.com/paper/2301.09506/) (2023)
- [Learning Attention as Disentangler for Compositional Zero-shot Learning](https://mlsanity.com/paper/2303.15111/) (2023)
- [Learning Conditional Attributes for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.17940/) (2023)
- [Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning](https://mlsanity.com/paper/2402.17251/) (2024)
- [Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2308.04016/) (2023)
- [Learning Clustering-based Prototypes for Compositional Zero-shot Learning](https://mlsanity.com/paper/2502.06501/) (2025)
- [Chop & Learn: Recognizing and Generating Object-State Compositions](https://mlsanity.com/paper/2309.14339/) (2023)
- [Distilled Reverse Attention Network for Open-world Compositional Zero-Shot Learning](https://mlsanity.com/paper/2303.00404/) (2023)
- [Composing Object Relations and Attributes for Image-Text Matching](https://mlsanity.com/paper/2406.11820/) (2024)
- [Multi-Task Learning of Object State Changes from Uncurated Videos](https://mlsanity.com/paper/2211.13500/) (2022)
- [ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2211.12417/) (2022)
- [C2C: Component-to-Composition Learning for Zero-Shot Compositional Action Recognition](https://mlsanity.com/paper/2407.06113/) (2024)
- [CusConcept: Customized Visual Concept Decomposition with Diffusion Models](https://mlsanity.com/paper/2410.00398/) (2024)
- [AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributes](https://mlsanity.com/paper/2305.14725/) (2023)
- [CAILA: Concept-Aware Intra-Layer Adapters for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.16681/) (2023)
- [TsCA: On the Semantic Consistency Alignment via Conditional Transport for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2408.08703/) (2024)
- [SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network](https://mlsanity.com/paper/2305.17624/) (2023)
- [On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Survey](https://mlsanity.com/paper/2408.04879/) (2024)
- [Zero-shot Compositional Action Recognition with Neural Logic Constraints](https://mlsanity.com/paper/2508.02320/) (2025)
- [Omni-Attribute: Open-vocabulary Attribute Encoder for Visual Concept Personalization](https://mlsanity.com/paper/2512.10955/) (2025)
- [Reference-Limited Compositional Zero-Shot Learning](https://mlsanity.com/paper/2208.10046/) (2022)
- [HOPE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts](https://mlsanity.com/paper/2311.14747/) (2023)
- [YOLOE-26: Integrating YOLO26 with YOLOE for Real-Time Open-Vocabulary Instance Segmentation](https://mlsanity.com/paper/2602.00168/) (2026)
- [Learning Concept-Based Causal Transition and Symbolic Reasoning for Visual Planning](https://mlsanity.com/paper/2310.03325/) (2023)
- [Anticipating Object State Changes in Long Procedural Videos](https://mlsanity.com/paper/2405.12789/) (2024)
- [Learning Visual Proxy for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2501.13859/) (2025)
- [A Conditional Probability Framework for Compositional Zero-shot Learning](https://mlsanity.com/paper/2507.17377/) (2025)
- [LOWA: Localize Objects in the Wild with Attributes](https://mlsanity.com/paper/2305.20047/) (2023)
- [Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs](https://mlsanity.com/paper/2307.12179/) (2023)

## Abstract (arXiv)

We study the problem of compositional zero-shot learning for object-attribute recognition. Prior works use visual features extracted with a backbone network, pre-trained for object classification and thus do not capture the subtly distinct features associated with attributes. To overcome this challenge, these studies employ supervision from the linguistic space, and use pre-trained word embeddings to better separate and compose attribute-object pairs for recognition. Analogous to linguistic embedding space, which already has unique and agnostic embeddings for object and attribute, we shift the focus back to the visual space and propose a novel architecture that can disentangle attribute and object features in the visual space. We use visual decomposed features to hallucinate embeddings that are representative for the seen and novel compositions to better regularize the learning of our model. Extensive experiments show that our method outperforms existing work with significant margin on three datasets: MIT-States, UT-Zappos, and a new benchmark created based on VAW. The code, models, and dataset splits are publicly available at https://github.com/nirat1606/OADis.