# HOPE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts

arXiv: https://arxiv.org/abs/2311.14747 · HTML: https://mlsanity.com/paper/2311.14747/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 34 of its 47 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [BASE Layers: Simplifying Training of Large, Sparse Models](https://mlsanity.com/paper/2103.16716/) (2021)
- [From Sparse to Soft Mixtures of Experts](https://mlsanity.com/paper/2308.00951/) (2023)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts](https://mlsanity.com/paper/2206.02770/) (2022)
- [Task-Driven Modular Networks for Zero-Shot Compositional Learning](https://mlsanity.com/paper/1905.05908/) (2019)
- [Learning Graph Embeddings for Compositional Zero-shot Learning](https://mlsanity.com/paper/2102.01987/) (2021)
- [Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2101.12609/) (2021)
- [Symmetry and Group in Attribute-Object Compositions](https://mlsanity.com/paper/2004.00587/) (2020)
- [Goal-Oriented Gaze Estimation for Zero-Shot Learning](https://mlsanity.com/paper/2103.03433/) (2021)
- [Hard Mixtures of Experts for Large Scale Weakly Supervised Vision](https://mlsanity.com/paper/1704.06363/) (2017)
- [Learning to Compose Soft Prompts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.03574/) (2022)
- [Disentangling Visual Embeddings for Attributes and Objects](https://mlsanity.com/paper/2205.08536/) (2022)
- [Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2206.14475/) (2022)
- [Learning Graph Embeddings for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2105.01017/) (2021)
- [Independent Prototype Propagation for Zero-Shot Compositionality](https://mlsanity.com/paper/2106.00305/) (2021)
- [KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2205.06784/) (2022)
- [DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning](https://mlsanity.com/paper/2207.01328/) (2022)
- [A Novel Perspective to Zero-shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion](https://mlsanity.com/paper/2004.14795/) (2020)
- [Learning Attention as Disentangler for Compositional Zero-shot Learning](https://mlsanity.com/paper/2303.15111/) (2023)
- [Learning Conditional Attributes for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.17940/) (2023)

## Built on by (most cited first, 3 of 3)

- [A Survey on Mixture of Experts in Large Language Models](https://mlsanity.com/paper/2407.06204/) (2024)
- [A Conditional Probability Framework for Compositional Zero-shot Learning](https://mlsanity.com/paper/2507.17377/) (2025)
- [Compositional Zero-Shot Learning: A Survey](https://mlsanity.com/paper/2510.11106/) (2025)

## Abstract (arXiv)

Compositional Zero-Shot Learning (CZSL) has emerged as an essential paradigm in machine learning, aiming to overcome the constraints of traditional zero-shot learning by incorporating compositional thinking into its methodology. Conventional zero-shot learning has difficulty managing unfamiliar combinations of seen and unseen classes because it depends on pre-defined class embeddings. In contrast, Compositional Zero-Shot Learning leverages the inherent hierarchies and structural connections among classes, creating new class representations by combining attributes, components, or other semantic elements. In our paper, we propose a novel framework that for the first time combines the Modern underlineHopfield Network with a Mixture underlineof underlineExunderlineperts (HOPE) to classify the compositions of previously unseen objects. Specifically, the Modern Hopfield Network creates a memory that stores label prototypes and identifies relevant labels for a given input image. Subsequently, the Mixture of Expert models integrates the image with the appropriate prototype to produce the final composition classification. Our approach achieves SOTA performance on several benchmarks, including MIT-States and UT-Zappos. We also examine how each component contributes to improved generalization.