# Contrastive language and vision learning of general fashion concepts

arXiv: https://arxiv.org/abs/2204.03972 · HTML: https://mlsanity.com/paper/2204.03972/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 109 references that are in the map.
- First posted on arXiv 8 Apr 2022.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [SLIP: Self-supervision meets Language-Image Pre-training](https://mlsanity.com/paper/2112.12750/) (2021)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning](https://mlsanity.com/paper/1810.08272/) (2018)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [Salient ImageNet: How to discover spurious features in Deep Learning?](https://mlsanity.com/paper/2110.04301/) (2021)
- [Learning Audio-Video Modalities from Image Captions](https://mlsanity.com/paper/2204.00679/) (2022)
- [Everything at Once -- Multi-modal Fusion Transformer for Video Retrieval](https://mlsanity.com/paper/2112.04446/) (2021)
- [Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks?](https://mlsanity.com/paper/2012.15180/) (2020)
- [Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition](https://mlsanity.com/paper/2109.06870/) (2021)
- [Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?](https://mlsanity.com/paper/2104.10809/) (2021)
- [Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models](https://mlsanity.com/paper/2005.10389/) (2020)
- [End-to-End Retrieval in Continuous Space](https://mlsanity.com/paper/1811.08008/) (2018)
- [Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others](https://mlsanity.com/paper/2102.11938/) (2021)
- [Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application](https://mlsanity.com/paper/1803.00710/) (2018)
- [ModaNet: A Large-Scale Street Fashion Dataset with Polygon Annotations](https://mlsanity.com/paper/1807.01394/) (2018)
- [Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation](https://mlsanity.com/paper/1607.07326/) (2016)
- [Fashion Meets Computer Vision: A Survey](https://mlsanity.com/paper/2003.13988/) (2020)
- [Reading Isn't Believing: Adversarial Attacks On Multi-Modal Neurons](https://mlsanity.com/paper/2103.10480/) (2021)
- [Interactive Grounded Language Acquisition and Generalization in a 2D World](https://mlsanity.com/paper/1802.01433/) (2018)

## Built on by (most cited first, 30 of 42)

- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [Garment3DGen: 3D Garment Stylization and Texture Generation](https://mlsanity.com/paper/2403.18816/) (2024)
- [BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity](https://mlsanity.com/paper/2406.12723/) (2024)
- [GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details](https://mlsanity.com/paper/2405.12420/) (2024)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [ViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Data](https://mlsanity.com/paper/2308.11194/) (2023)
- [OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Data](https://mlsanity.com/paper/2309.05551/) (2023)
- [MOON: Generative MLLM-based Multimodal Representation Learning for E-commerce Product Understanding](https://mlsanity.com/paper/2508.11999/) (2025)
- [Captions Speak Louder than Images: Generalizing Foundation Models for E-commerce from High-quality Multimodal Instruction Data](https://mlsanity.com/paper/2410.17337/) (2024)
- [Reasonable Scale Machine Learning with Open-Source Metaflow](https://mlsanity.com/paper/2303.11761/) (2023)
- [Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model](https://mlsanity.com/paper/2404.11046/) (2024)
- [Open-Vocabulary Semantic Part Segmentation of 3D Human](https://mlsanity.com/paper/2502.19782/) (2025)
- [EcomMMMU: Strategic Utilization of Visuals for Robust Multimodal E-commerce Models](https://mlsanity.com/paper/2508.15721/) (2025)
- [MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding](https://mlsanity.com/paper/2511.12449/) (2025)
- [Seeing the Abstract: Translating the Abstract Language for Vision Language Models](https://mlsanity.com/paper/2505.03242/) (2025)
- [Can GPT-4o mini and Gemini 2.0 Flash Predict Fine-Grained Fashion Product Attributes? A Zero-Shot Analysis](https://mlsanity.com/paper/2507.09950/) (2025)
- [Delving into E-Commerce Product Retrieval with Vision-Language Pre-training](https://mlsanity.com/paper/2304.04377/) (2023)
- [TriplePlay: Enhancing Federated Learning with CLIP for Non-IID Data and Resource Efficiency](https://mlsanity.com/paper/2409.05347/) (2024)
- [A Text-to-3D Framework for Joint Generation of CG-Ready Humans and Compatible Garments](https://mlsanity.com/paper/2503.12052/) (2025)
- [TATTOO: Training-free AesTheTic-aware Outfit recOmmendation](https://mlsanity.com/paper/2509.23242/) (2025)
- [CorrEmbed: Evaluating Pre-trained Model Image Similarity Efficacy with a Novel Metric](https://mlsanity.com/paper/2308.16126/) (2023)
- [AI Recommendation System for Enhanced Customer Experience: A Novel Image-to-Text Method](https://mlsanity.com/paper/2311.09624/) (2023)
- [Assessing Graphical Perception of Image Embedding Models using Channel Effectiveness](https://mlsanity.com/paper/2407.20845/) (2024)
- [Zero-Shot Product Attribute Labeling with Vision-Language Models: A Three-Tier Evaluation Framework](https://mlsanity.com/paper/2601.15711/) (2026)
- [ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval](https://mlsanity.com/paper/2606.27708/) (2026)
- [(Vector) Space is Not the Final Frontier: Product Search as Program Synthesis](https://mlsanity.com/paper/2304.11473/) (2023)
- [Dataset Factory: A Toolchain For Generative Computer Vision Datasets](https://mlsanity.com/paper/2309.11608/) (2023)
- [Fashion Style Editing with Generative Human Prior](https://mlsanity.com/paper/2404.01984/) (2024)
- [Reflective Human-Machine Co-adaptation for Enhanced Text-to-Image Generation Dialogue System](https://mlsanity.com/paper/2409.07464/) (2024)
- [ENCLIP: Ensembling and Clustering-Based Contrastive Language-Image Pretraining for Fashion Multimodal Search with Limited Data and Low-Quality Images](https://mlsanity.com/paper/2411.16096/) (2024)

## Abstract (arXiv)

The steady rise of online shopping goes hand in hand with the development of increasingly complex ML and NLP models. While most use cases are cast as specialized supervised learning problems, we argue that practitioners would greatly benefit from more transferable representations of products. In this work, we build on recent developments in contrastive learning to train FashionCLIP, a CLIP-like model for the fashion industry. We showcase its capabilities for retrieval, classification and grounding, and release our model and code to the community.