# Learning Fashion Compatibility with Bidirectional LSTMs

arXiv: https://arxiv.org/abs/1707.05691 · HTML: https://mlsanity.com/paper/1707.05691/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2017.

## Built on by (most cited first, 30 of 69)

- [GeneCIS: A Benchmark for General Conditional Image Similarity](https://mlsanity.com/paper/2306.07969/) (2023)
- [Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback](https://mlsanity.com/paper/2106.04128/) (2021)
- [OutfitTransformer: Learning Outfit Representations for Fashion Recommendation](https://mlsanity.com/paper/2204.04812/) (2022)
- [SGDiff: A Style Guided Diffusion Model for Fashion Synthesis](https://mlsanity.com/paper/2308.07605/) (2023)
- [FashionSAP: Symbols and Attributes Prompt for Fine-grained Fashion Vision-Language Pre-training](https://mlsanity.com/paper/2304.05051/) (2023)
- [Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning](https://mlsanity.com/paper/2104.02429/) (2021)
- [Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training](https://mlsanity.com/paper/2108.11119/) (2021)
- [Effectively Leveraging Attributes for Visual Similarity](https://mlsanity.com/paper/2105.01695/) (2021)
- [Fine-tuning Multimodal Large Language Models for Product Bundling](https://mlsanity.com/paper/2407.11712/) (2024)
- [Semi-Supervised Visual Representation Learning for Fashion Compatibility](https://mlsanity.com/paper/2109.08052/) (2021)
- [From Culture to Clothing: Discovering the World Events Behind A Century of Fashion Images](https://mlsanity.com/paper/2102.01690/) (2021)
- [UIGR: Unified Interactive Garment Retrieval](https://mlsanity.com/paper/2204.03111/) (2022)
- [DM-VTON: Distilled Mobile Real-time Virtual Try-On](https://mlsanity.com/paper/2308.13798/) (2023)
- [Limb-Aware Virtual Try-On Network with Progressive Clothing Warping](https://mlsanity.com/paper/2503.14074/) (2025)
- [Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting](https://mlsanity.com/paper/2101.08301/) (2021)
- [I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion](https://mlsanity.com/paper/2102.08871/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing](https://mlsanity.com/paper/2111.12346/) (2021)
- [Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework](https://mlsanity.com/paper/2502.06827/) (2025)
- [Learning Shared Representations from Unpaired Data](https://mlsanity.com/paper/2505.21524/) (2025)
- [Garments2Look: A Multi-Reference Dataset for High-Fidelity Outfit-Level Virtual Try-On with Clothing and Accessories](https://mlsanity.com/paper/2603.14153/) (2026)
- [SAT: Self-adaptive training for fashion compatibility prediction](https://mlsanity.com/paper/2206.12622/) (2022)
- [Fashion Recommendation Based on Style and Social Events](https://mlsanity.com/paper/2208.00725/) (2022)
- [Attribute-Guided Multi-Level Attention Network for Fine-Grained Fashion Retrieval](https://mlsanity.com/paper/2301.13014/) (2022)
- [Transformer-based Graph Neural Networks for Outfit Generation](https://mlsanity.com/paper/2304.08098/) (2023)
- [Hierarchical Matching and Reasoning for Multi-Query Image Retrieval](https://mlsanity.com/paper/2306.14460/) (2023)
- [Efficient Discovery and Effective Evaluation of Visual Perceptual Similarity: A Benchmark and Beyond](https://mlsanity.com/paper/2308.14753/) (2023)
- [Progressive Limb-Aware Virtual Try-On](https://mlsanity.com/paper/2503.12588/) (2025)
- [SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport](https://mlsanity.com/paper/2602.23353/) (2026)
- [Attribute-aware Explainable Complementary Clothing Recommendation](https://mlsanity.com/paper/2107.01655/) (2021)

## Abstract (arXiv)

The ubiquity of online fashion shopping demands effective recommendation services for customers. In this paper, we study two types of fashion recommendation: (i) suggesting an item that matches existing components in a set to form a stylish outfit (a collection of fashion items), and (ii) generating an outfit with multimodal (images/text) specifications from a user. To this end, we propose to jointly learn a visual-semantic embedding and the compatibility relationships among fashion items in an end-to-end fashion. More specifically, we consider a fashion outfit to be a sequence (usually from top to bottom and then accessories) and each item in the outfit as a time step. Given the fashion items in an outfit, we train a bidirectional LSTM (Bi-LSTM) model to sequentially predict the next item conditioned on previous ones to learn their compatibility relationships. Further, we learn a visual-semantic space by regressing image features to their semantic representations aiming to inject attribute and category information as a regularization for training the LSTM. The trained network can not only perform the aforementioned recommendations effectively but also predict the compatibility of a given outfit. We conduct extensive experiments on our newly collected Polyvore dataset, and the results provide strong qualitative and quantitative evidence that our framework outperforms alternative methods.