# Who, Where, and What to Wear? Extracting Fashion Knowledge from Social Media

arXiv: https://arxiv.org/abs/1908.08985 · HTML: https://mlsanity.com/paper/1908.08985/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2019.

## Built on by (most cited first, 10 of 10)

- [Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning](https://mlsanity.com/paper/2104.02429/) (2021)
- [UIGR: Unified Interactive Garment Retrieval](https://mlsanity.com/paper/2204.03111/) (2022)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Large Scale Long-tailed Product Recognition System at Alibaba](https://mlsanity.com/paper/2102.04652/) (2021)
- [Leveraging Multiple Relations for Fashion Trend Forecasting Based on Social Media](https://mlsanity.com/paper/2105.03299/) (2021)
- [Sentiment Analysis of Fashion Related Posts in Social Media](https://mlsanity.com/paper/2111.07815/) (2021)
- [Borrowing Human Senses: Comment-Aware Self-Training for Social Media Multimodal Classification](https://mlsanity.com/paper/2303.15016/) (2023)
- [fAshIon after fashion: A Report of AI in Fashion](https://mlsanity.com/paper/2105.03050/) (2021)
- [Social Media Fashion Knowledge Extraction as Captioning](https://mlsanity.com/paper/2309.16270/) (2023)
- [FashionKG-RAG: Knowledge Graph-Enhanced Retrieval-Augmented Generation for Fashion Question Answering](https://mlsanity.com/paper/2608.22688/) (2026)

## Abstract (arXiv)

Fashion knowledge helps people to dress properly and addresses not only physiological needs of users, but also the demands of social activities and conventions. It usually involves three mutually related aspects of: occasion, person and clothing. However, there are few works focusing on extracting such knowledge, which will greatly benefit many downstream applications, such as fashion recommendation. In this paper, we propose a novel method to automatically harvest fashion knowledge from social media. We unify three tasks of occasion, person and clothing discovery from multiple modalities of images, texts and metadata. For person detection and analysis, we use the off-the-shelf tools due to their flexibility and satisfactory performance. For clothing recognition and occasion prediction, we unify the two tasks by using a contextualized fashion concept learning module, which captures the dependencies and correlations among different fashion concepts. To alleviate the heavy burden of human annotations, we introduce a weak label modeling module which can effectively exploit machine-labeled data, a complementary of clean data. In experiments, we contribute a benchmark dataset and conduct extensive experiments from both quantitative and qualitative perspectives. The results demonstrate the effectiveness of our model in fashion concept prediction, and the usefulness of extracted knowledge with comprehensive analysis.