# Visually-Aware Fashion Recommendation and Design with Generative Image Models

arXiv: https://arxiv.org/abs/1711.02231 · HTML: https://mlsanity.com/paper/1711.02231/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2017.

## Built on by (most cited first, 30 of 43)

- [StyleGAN-Human: A Data-Centric Odyssey of Human Generation](https://mlsanity.com/paper/2204.11823/) (2022)
- [Mining Latent Structures for Multimedia Recommendation](https://mlsanity.com/paper/2104.09036/) (2021)
- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-modal Pretraining](https://mlsanity.com/paper/2107.14572/) (2021)
- [Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfer](https://mlsanity.com/paper/2307.07754/) (2023)
- [Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond](https://mlsanity.com/paper/2403.10667/) (2024)
- [Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation](https://mlsanity.com/paper/2106.06165/) (2021)
- [A Survey on Diffusion Models for Recommender Systems](https://mlsanity.com/paper/2409.05033/) (2024)
- [Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendation](https://mlsanity.com/paper/2111.00678/) (2021)
- [Multimodality Invariant Learning for Multimedia-Based New Item Recommendation](https://mlsanity.com/paper/2405.15783/) (2024)
- [Debiasing Learning for Membership Inference Attacks Against Recommender Systems](https://mlsanity.com/paper/2206.12401/) (2022)
- [Effectively Leveraging Attributes for Visual Similarity](https://mlsanity.com/paper/2105.01695/) (2021)
- [A Fashion Item Recommendation Model in Hyperbolic Space](https://mlsanity.com/paper/2409.02599/) (2024)
- [Sequential LLM Framework for Fashion Recommendation](https://mlsanity.com/paper/2410.11327/) (2024)
- [Structured Spectral Reasoning for Frequency-Adaptive Multimodal Recommendation](https://mlsanity.com/paper/2512.01372/) (2025)
- [MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling](https://mlsanity.com/paper/2512.07216/) (2025)
- [End-to-End Image-Based Fashion Recommendation](https://mlsanity.com/paper/2205.02923/) (2022)
- [Mining Stable Preferences: Adaptive Modality Decorrelation for Multimedia Recommendation](https://mlsanity.com/paper/2306.14179/) (2023)
- [Adversarial Item Promotion on Visually-Aware Recommender Systems by Guided Diffusion](https://mlsanity.com/paper/2312.15826/) (2023)
- [Modality-Balanced Learning for Multimedia Recommendation](https://mlsanity.com/paper/2408.06360/) (2024)
- [Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting](https://mlsanity.com/paper/2101.08301/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [LaViC: Adapting Large Vision-Language Models to Visually-Aware Conversational Recommendation](https://mlsanity.com/paper/2503.23312/) (2025)
- [Pareto Invariant Representation Learning for Multimedia Recommendation](https://mlsanity.com/paper/2308.04706/) (2023)
- [Sentiment Analysis of Fashion Related Posts in Social Media](https://mlsanity.com/paper/2111.07815/) (2021)
- [Docent: A content-based recommendation system to discover contemporary art](https://mlsanity.com/paper/2207.05648/) (2022)
- [fAshIon after fashion: A Report of AI in Fashion](https://mlsanity.com/paper/2105.03050/) (2021)
- [Securing Visually-Aware Recommender Systems: An Adversarial Image Reconstruction and Detection Framework](https://mlsanity.com/paper/2306.07992/) (2023)
- [Social Media Fashion Knowledge Extraction as Captioning](https://mlsanity.com/paper/2309.16270/) (2023)
- [Lost Your Style? Navigating with Semantic-Level Approach for Text-to-Outfit Retrieval](https://mlsanity.com/paper/2311.02122/) (2023)

## Abstract (arXiv)

Building effective recommender systems for domains like fashion is challenging due to the high level of subjectivity and the semantic complexity of the features involved (i.e., fashion styles). Recent work has shown that approaches to `visual' recommendation (e.g.~clothing, art, etc.) can be made more accurate by incorporating visual signals directly into the recommendation objective, using `off-the-shelf' feature representations derived from deep networks. Here, we seek to extend this contribution by showing that recommendation performance can be significantly improved by learning `fashion aware' image representations directly, i.e., by training the image representation (from the pixel level) and the recommender system jointly; this contribution is related to recent work using Siamese CNNs, though we are able to show improvements over state-of-the-art recommendation techniques such as BPR and variants that make use of pre-trained visual features. Furthermore, we show that our model can be used generatively, i.e., given a user and a product category, we can generate new images (i.e., clothing items) that are most consistent with their personal taste. This represents a first step towards building systems that go beyond recommending existing items from a product corpus, but which can be used to suggest styles and aid the design of new products.