# Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data

arXiv: https://arxiv.org/abs/1608.03016 · HTML: https://mlsanity.com/paper/1608.03016/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2016.

## Built on by (most cited first, 16 of 16)

- [Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback](https://mlsanity.com/paper/2106.04128/) (2021)
- [Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning](https://mlsanity.com/paper/2104.02429/) (2021)
- [Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting](https://mlsanity.com/paper/2101.08301/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing](https://mlsanity.com/paper/2111.12346/) (2021)
- [Leveraging Multiple Relations for Fashion Trend Forecasting Based on Social Media](https://mlsanity.com/paper/2105.03299/) (2021)
- [Attribute-aware Explainable Complementary Clothing Recommendation](https://mlsanity.com/paper/2107.01655/) (2021)
- [Sentiment Analysis of Fashion Related Posts in Social Media](https://mlsanity.com/paper/2111.07815/) (2021)
- [COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles](https://mlsanity.com/paper/2502.08674/) (2025)
- [Imitation Learning for Fashion Style Based on Hierarchical Multimodal Representation](https://mlsanity.com/paper/2004.06229/) (2020)
- [Visual analytics of set data for knowledge discovery and member selection support](https://mlsanity.com/paper/2104.09231/) (2021)
- [Main Product Detection with Graph Networks for Fashion](https://mlsanity.com/paper/2201.10431/) (2022)
- [Partial Visual-Semantic Embedding: Fashion Intelligence System with Sensitive Part-by-Part Learning](https://mlsanity.com/paper/2211.06688/) (2022)
- [ICAR: Image-based Complementary Auto Reasoning](https://mlsanity.com/paper/2308.09119/) (2023)
- [A Hybrid Multimodal Deep Learning Framework for Intelligent Fashion Recommendation](https://mlsanity.com/paper/2511.07573/) (2025)
- [Recommending Composite Items Using Multi-Level Preference Information: A Joint Interaction Modeling Approach](https://mlsanity.com/paper/2601.19005/) (2026)

## Abstract (arXiv)

Composing fashion outfits involves deep understanding of fashion standards while incorporating creativity for choosing multiple fashion items (e.g., Jewelry, Bag, Pants, Dress). In fashion websites, popular or high-quality fashion outfits are usually designed by fashion experts and followed by large audiences. In this paper, we propose a machine learning system to compose fashion outfits automatically. The core of the proposed automatic composition system is to score fashion outfit candidates based on the appearances and meta-data. We propose to leverage outfit popularity on fashion oriented websites to supervise the scoring component. The scoring component is a multi-modal multi-instance deep learning system that evaluates instance aesthetics and set compatibility simultaneously. In order to train and evaluate the proposed composition system, we have collected a large scale fashion outfit dataset with 195K outfits and 368K fashion items from Polyvore. Although the fashion outfit scoring and composition is rather challenging, we have achieved an AUC of 85% for the scoring component, and an accuracy of 77% for a constrained composition task.