# Complete the Look: Scene-based Complementary Product Recommendation

arXiv: https://arxiv.org/abs/1812.01748 · HTML: https://mlsanity.com/paper/1812.01748/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2018.

## Built on by (most cited first, 21 of 21)

- [Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-modal Pretraining](https://mlsanity.com/paper/2107.14572/) (2021)
- [From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval](https://mlsanity.com/paper/2305.10260/) (2023)
- [AdBooster: Personalized Ad Creative Generation using Stable Diffusion Outpainting](https://mlsanity.com/paper/2309.11507/) (2023)
- [ProS: Prompting-to-simulate Generalized knowledge for Universal Cross-Domain Retrieval](https://mlsanity.com/paper/2312.12478/) (2023)
- [Towards High-Order Complementary Recommendation via Logical Reasoning Network](https://mlsanity.com/paper/2212.04966/) (2022)
- [CCGen: Explainable Complementary Concept Generation in E-Commerce](https://mlsanity.com/paper/2305.11480/) (2023)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Attribute-aware Explainable Complementary Clothing Recommendation](https://mlsanity.com/paper/2107.01655/) (2021)
- [FashionSearchNet-v2: Learning Attribute Representations with Localization for Image Retrieval with Attribute Manipulation](https://mlsanity.com/paper/2111.14145/) (2021)
- [fAshIon after fashion: A Report of AI in Fashion](https://mlsanity.com/paper/2105.03050/) (2021)
- [Efficient Large-Scale Visual Representation Learning And Evaluation](https://mlsanity.com/paper/2305.13399/) (2023)
- [Lost Your Style? Navigating with Semantic-Level Approach for Text-to-Outfit Retrieval](https://mlsanity.com/paper/2311.02122/) (2023)
- [Character-based Outfit Generation with Vision-augmented Style Extraction via LLMs](https://mlsanity.com/paper/2402.05941/) (2024)
- [Complementary Recommendation in E-commerce: Definition, Approaches, and Future Directions](https://mlsanity.com/paper/2403.16135/) (2024)
- [FashionFail: Addressing Failure Cases in Fashion Object Detection and Segmentation](https://mlsanity.com/paper/2404.08582/) (2024)
- [Automatic Scene-based Topic Channel Construction System for E-Commerce](https://mlsanity.com/paper/2210.02643/) (2022)
- [Partial Visual-Semantic Embedding: Fashion Intelligence System with Sensitive Part-by-Part Learning](https://mlsanity.com/paper/2211.06688/) (2022)
- [ICAR: Image-based Complementary Auto Reasoning](https://mlsanity.com/paper/2308.09119/) (2023)
- [A Survey of Reasoning for Substitution Relationships: Definitions, Methods, and Directions](https://mlsanity.com/paper/2404.08687/) (2024)
- [Revisiting Graph Projections for Effective Complementary Product Recommendation](https://mlsanity.com/paper/2506.09209/) (2025)
- [Optimizing Product Deduplication in E-Commerce with Multimodal Embeddings](https://mlsanity.com/paper/2509.15858/) (2025)

## Abstract (arXiv)

Modeling fashion compatibility is challenging due to its complexity and subjectivity. Existing work focuses on predicting compatibility between product images (e.g. an image containing a t-shirt and an image containing a pair of jeans). However, these approaches ignore real-world 'scene' images (e.g. selfies); such images are hard to deal with due to their complexity, clutter, variations in lighting and pose (etc.) but on the other hand could potentially provide key context (e.g. the user's body type, or the season) for making more accurate recommendations. In this work, we propose a new task called 'Complete the Look', which seeks to recommend visually compatible products based on scene images. We design an approach to extract training data for this task, and propose a novel way to learn the scene-product compatibility from fashion or interior design images. Our approach measures compatibility both globally and locally via CNNs and attention mechanisms. Extensive experiments show that our method achieves significant performance gains over alternative systems. Human evaluation and qualitative analysis are also conducted to further understand model behavior. We hope this work could lead to useful applications which link large corpora of real-world scenes with shoppable products.