# Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning

arXiv: https://arxiv.org/abs/2308.04016 · HTML: https://mlsanity.com/paper/2308.04016/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 83 references that are in the map.
- First posted on arXiv 8 Aug 2023.

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](https://mlsanity.com/paper/2009.12991/) (2020)
- [Cascaded Partial Decoder for Fast and Accurate Salient Object Detection](https://mlsanity.com/paper/1904.08739/) (2019)
- [Improving Calibration for Long-Tailed Recognition](https://mlsanity.com/paper/2104.00466/) (2021)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild](https://mlsanity.com/paper/1605.04253/) (2016)

## Built on by (most cited first, 16 of 16)

- [Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning](https://mlsanity.com/paper/2402.17251/) (2024)
- [Learning Clustering-based Prototypes for Compositional Zero-shot Learning](https://mlsanity.com/paper/2502.06501/) (2025)
- [Compositional Zero-shot Learning via Progressive Language-based Observations](https://mlsanity.com/paper/2311.14749/) (2023)
- [C2C: Component-to-Composition Learning for Zero-Shot Compositional Action Recognition](https://mlsanity.com/paper/2407.06113/) (2024)
- [On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Survey](https://mlsanity.com/paper/2408.04879/) (2024)
- [HOPE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts](https://mlsanity.com/paper/2311.14747/) (2023)
- [FlowComposer: Composable Flows for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2603.16641/) (2026)
- [Not Just Object, But State: Compositional Incremental Learning without Forgetting](https://mlsanity.com/paper/2411.01739/) (2024)
- [Learning Visual Proxy for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2501.13859/) (2025)
- [A Conditional Probability Framework for Compositional Zero-shot Learning](https://mlsanity.com/paper/2507.17377/) (2025)
- [Unified Framework for Open-World Compositional Zero-shot Learning](https://mlsanity.com/paper/2412.04083/) (2024)
- [SEER-ZSL: Semantic Encoder-Enhanced Representations for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2312.13100/) (2023)
- [Semantically Guided Dynamic Visual Prototype Refinement for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2501.07114/) (2025)
- [Why Can't I Open My Drawer? Mitigating Object-Driven Shortcuts in Zero-Shot Compositional Action Recognition](https://mlsanity.com/paper/2601.16211/) (2026)
- [Progressive Reasoning with Primitive Correction for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2607.05911/) (2026)
- [DIFFCZSL: Compositional Zero-Shot Learning Regularized by Diffusion Representations](https://mlsanity.com/paper/2608.19871/) (2026)

## Abstract (arXiv)

Compositional zero-shot learning (CZSL) aims to recognize unseen compositions with prior knowledge of known primitives (attribute and object). Previous works for CZSL often suffer from grasping the contextuality between attribute and object, as well as the discriminability of visual features, and the long-tailed distribution of real-world compositional data. We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. CoT employs object and attribute experts in distinctive manners to generate representative embeddings, using the visual network hierarchically. The object expert extracts representative object embeddings from the final layer in a bottom-up manner, while the attribute expert makes attribute embeddings in a top-down manner with a proposed object-guided attention module that models contextuality explicitly. To remedy biased prediction caused by imbalanced data distribution, we develop a simple minority attribute augmentation (MAA) that synthesizes virtual samples by mixing two images and oversampling minority attribute classes. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL. We also demonstrate the effectiveness of CoT in improving visual discrimination and addressing the model bias from the imbalanced data distribution. The code is available at https://github.com/HanjaeKim98/CoT.