# OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Data

arXiv: https://arxiv.org/abs/2309.05551 · HTML: https://mlsanity.com/paper/2309.05551/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm](https://mlsanity.com/paper/2110.05208/) (2021)
- [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023)
- [Dress Code: High-Resolution Multi-Category Virtual Try-On](https://mlsanity.com/paper/2204.08532/) (2022)
- [LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On](https://mlsanity.com/paper/2305.13501/) (2023)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions](https://mlsanity.com/paper/2206.14180/) (2022)
- [GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning](https://mlsanity.com/paper/2303.13756/) (2023)
- [Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On](https://mlsanity.com/paper/2105.06462/) (2021)
- [Kaleido-BERT: Vision-Language Pre-training on Fashion Domain](https://mlsanity.com/paper/2103.16110/) (2021)
- [Fashion-Gen: The Generative Fashion Dataset and Challenge](https://mlsanity.com/paper/1806.08317/) (2018)
- [Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing](https://mlsanity.com/paper/2304.02051/) (2023)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [The iMaterialist Fashion Attribute Dataset](https://mlsanity.com/paper/1906.05750/) (2019)
- [FashionViL: Fashion-Focused Vision-and-Language Representation Learning](https://mlsanity.com/paper/2207.08150/) (2022)
- [Fashion Retrieval via Graph Reasoning Networks on a Similarity Pyramid](https://mlsanity.com/paper/1908.11754/) (2019)
- [Visual Search at Alibaba](https://mlsanity.com/paper/2102.04674/) (2021)
- [Context-Aware Visual Compatibility Prediction](https://mlsanity.com/paper/1902.03646/) (2019)
- [Learning a Unified Embedding for Visual Search at Pinterest](https://mlsanity.com/paper/1908.01707/) (2019)
- [Creating Capsule Wardrobes from Fashion Images](https://mlsanity.com/paper/1712.02662/) (2017)
- [OutfitTransformer: Learning Outfit Representations for Fashion Recommendation](https://mlsanity.com/paper/2204.04812/) (2022)

## Built on by (most cited first, 9 of 9)

- [Test-time Adaptation for Cross-modal Retrieval with Query Shift](https://mlsanity.com/paper/2410.15624/) (2024)
- [Learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Engine](https://mlsanity.com/paper/2312.15844/) (2023)
- [Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing](https://mlsanity.com/paper/2403.14828/) (2024)
- [Open-Vocabulary Semantic Part Segmentation of 3D Human](https://mlsanity.com/paper/2502.19782/) (2025)
- [Seeing the Abstract: Translating the Abstract Language for Vision Language Models](https://mlsanity.com/paper/2505.03242/) (2025)
- [AnyDesign: Versatile Area Fashion Editing via Mask-Free Diffusion](https://mlsanity.com/paper/2408.11553/) (2024)
- [Agentic Personalized Fashion Recommendation in the Age of Generative AI: Challenges, Opportunities, and Evaluation](https://mlsanity.com/paper/2508.02342/) (2025)
- [Performance-Efficiency Trade-off for Fashion Image Retrieval](https://mlsanity.com/paper/2509.24477/) (2025)
- [OmniFashion: Towards Generalist Fashion Intelligence via Multi-Task Vision-Language Learning](https://mlsanity.com/paper/2603.02658/) (2026)

## Abstract (arXiv)

The inexorable growth of online shopping and e-commerce demands scalable and robust machine learning-based solutions to accommodate customer requirements. In the context of automatic tagging classification and multimodal retrieval, prior works either defined a low generalizable supervised learning approach or more reusable CLIP-based techniques while, however, training on closed source data. In this work, we propose OpenFashionCLIP, a vision-and-language contrastive learning method that only adopts open-source fashion data stemming from diverse domains, and characterized by varying degrees of specificity. Our approach is extensively validated across several tasks and benchmarks, and experimental results highlight a significant out-of-domain generalization capability and consistent improvements over state-of-the-art methods both in terms of accuracy and recall. Source code and trained models are publicly available at: https://github.com/aimagelab/open-fashion-clip.