# "Does it come in black?" CLIP-like models are zero-shot recommenders

arXiv: https://arxiv.org/abs/2204.02473 · HTML: https://mlsanity.com/paper/2204.02473/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [A Survey on Conversational Recommender Systems](https://mlsanity.com/paper/2004.00646/) (2020)
- [ExprGAN: Facial Expression Editing with Controllable Expression Intensity](https://mlsanity.com/paper/1709.03842/) (2017)
- [Product Knowledge Graph Embedding for E-commerce](https://mlsanity.com/paper/1911.12481/) (2019)
- [Try This Instead: Personalized and Interpretable Substitute Recommendation](https://mlsanity.com/paper/2005.09344/) (2020)
- [Generate Natural Language Explanations for Recommendation](https://mlsanity.com/paper/2101.03392/) (2021)
- [Fantastic Embeddings and How to Align Them: Zero-Shot Inference in a Multi-Shop Scenario](https://mlsanity.com/paper/2007.14906/) (2020)
- [SIGIR 2021 E-Commerce Workshop Data Challenge](https://mlsanity.com/paper/2104.09423/) (2021)
- [Query2Prod2Vec Grounded Word Embeddings for eCommerce](https://mlsanity.com/paper/2104.02061/) (2021)
- [BERT Goes Shopping: Comparing Distributional Models for Product Representations](https://mlsanity.com/paper/2012.09807/) (2020)
- [Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction](https://mlsanity.com/paper/2104.08874/) (2021)
- [A Multimodal Recommender System for Large-scale Assortment Generation in E-commerce](https://mlsanity.com/paper/1806.11226/) (2018)
- [Personalized Entity Resolution with Dynamic Heterogeneous Knowledge Graph Representations](https://mlsanity.com/paper/2104.02667/) (2021)
- ["Are you sure?": Preliminary Insights from Scaling Product Comparisons to Multiple Shops](https://mlsanity.com/paper/2107.03256/) (2021)
- [On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems](https://mlsanity.com/paper/1907.07629/) (2019)

## Built on by (most cited first, 4 of 4)

- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [Reasonable Scale Machine Learning with Open-Source Metaflow](https://mlsanity.com/paper/2303.11761/) (2023)
- [(Vector) Space is Not the Final Frontier: Product Search as Program Synthesis](https://mlsanity.com/paper/2304.11473/) (2023)
- [ENCLIP: Ensembling and Clustering-Based Contrastive Language-Image Pretraining for Fashion Multimodal Search with Limited Data and Low-Quality Images](https://mlsanity.com/paper/2411.16096/) (2024)

## Abstract (arXiv)

Product discovery is a crucial component for online shopping. However, item-to-item recommendations today do not allow users to explore changes along selected dimensions: given a query item, can a model suggest something similar but in a different color? We consider item recommendations of the comparative nature (e.g. "something darker") and show how CLIP-based models can support this use case in a zero-shot manner. Leveraging a large model built for fashion, we introduce GradREC and its industry potential, and offer a first rounded assessment of its strength and weaknesses.