# Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation

arXiv: https://arxiv.org/abs/1806.08977 · HTML: https://mlsanity.com/paper/1806.08977/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2018.

## Built on by (most cited first, 16 of 16)

- [Multimodal Recommender Systems: A Survey](https://mlsanity.com/paper/2302.03883/) (2023)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions](https://mlsanity.com/paper/2306.05731/) (2023)
- [A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future](https://mlsanity.com/paper/2412.14056/) (2024)
- [Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting](https://mlsanity.com/paper/2101.08301/) (2021)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Deciphering Compatibility Relationships with Textual Descriptions via Extraction and Explanation](https://mlsanity.com/paper/2312.11554/) (2023)
- [Transformer-based Graph Neural Networks for Outfit Generation](https://mlsanity.com/paper/2304.08098/) (2023)
- [Attribute-aware Explainable Complementary Clothing Recommendation](https://mlsanity.com/paper/2107.01655/) (2021)
- [Visualization for Recommendation Explainability: A Survey and New Perspectives](https://mlsanity.com/paper/2305.11755/) (2023)
- [FashionM3: Multimodal, Multitask, and Multiround Fashion Assistant based on Unified Vision-Language Model](https://mlsanity.com/paper/2504.17826/) (2025)
- [TATTOO: Training-free AesTheTic-aware Outfit recOmmendation](https://mlsanity.com/paper/2509.23242/) (2025)
- [BiVRec: Bidirectional View-based Multimodal Sequential Recommendation](https://mlsanity.com/paper/2402.17334/) (2024)
- [Recommendation of Compatible Outfits Conditioned on Style](https://mlsanity.com/paper/2203.16161/) (2022)
- [Visual or Textual: Effects of Explanation Format and Personal Characteristics on the Perception of Explanations in an Educational Recommender System](https://mlsanity.com/paper/2603.25624/) (2026)
- [Grounding Free-Form Instructions for Fashion Complementary Image Generation](https://mlsanity.com/paper/2608.23302/) (2026)

## Abstract (arXiv)

Most previous work on outfit recommendation focuses on designing visual features to enhance recommendations. Existing work neglects user comments of fashion items, which have been proved to be effective in generating explanations along with better recommendation results. We propose a novel neural network framework, neural outfit recommendation (NOR), that simultaneously provides outfit recommendations and generates abstractive comments. NOR consists of two parts: outfit matching and comment generation. For outfit matching, we propose a convolutional neural network with a mutual attention mechanism to extract visual features. The visual features are then decoded into a rating score for the matching prediction. For abstractive comment generation, we propose a gated recurrent neural network with a cross-modality attention mechanism to transform visual features into a concise sentence. The two parts are jointly trained based on a multi-task learning framework in an end-to-end back-propagation paradigm. Extensive experiments conducted on an existing dataset and a collected real-world dataset show NOR achieves significant improvements over state-of-the-art baselines for outfit recommendation. Meanwhile, our generated comments achieve impressive ROUGE and BLEU scores in comparison to human-written comments. The generated comments can be regarded as explanations for the recommendation results. We release the dataset and code to facilitate future research.