# Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback

arXiv: https://arxiv.org/abs/2106.04128 · HTML: https://mlsanity.com/paper/2106.04128/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Embodied Question Answering](https://mlsanity.com/paper/1711.11543/) (2017)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [Dialog-based Interactive Image Retrieval](https://mlsanity.com/paper/1805.00145/) (2018)
- [Boosting Image Captioning with Attributes](https://mlsanity.com/paper/1611.01646/) (2016)
- [Learning Fashion Compatibility with Bidirectional LSTMs](https://mlsanity.com/paper/1707.05691/) (2017)
- [Fashion-Gen: The Generative Fashion Dataset and Challenge](https://mlsanity.com/paper/1806.08317/) (2018)
- [Compositional Learning of Image-Text Query for Image Retrieval](https://mlsanity.com/paper/2006.11149/) (2020)
- [Image Captioning and Visual Question Answering Based on Attributes and External Knowledge](https://mlsanity.com/paper/1603.02814/) (2016)
- [Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction](https://mlsanity.com/paper/1511.05756/) (2015)
- [Fashion Forward: Forecasting Visual Style in Fashion](https://mlsanity.com/paper/1705.06394/) (2017)
- [Creating Capsule Wardrobes from Fashion Images](https://mlsanity.com/paper/1712.02662/) (2017)
- [CurlingNet: Compositional Learning between Images and Text for Fashion IQ Data](https://mlsanity.com/paper/2003.12299/) (2020)
- [Learning the Latent "Look": Unsupervised Discovery of a Style-Coherent Embedding from Fashion Images](https://mlsanity.com/paper/1707.03376/) (2017)
- [Explainable Fashion Recommendation: A Semantic Attribute Region Guided Approach](https://mlsanity.com/paper/1905.12862/) (2019)
- [Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data](https://mlsanity.com/paper/1608.03016/) (2016)
- [Aesthetic-based Clothing Recommendation](https://mlsanity.com/paper/1809.05822/) (2018)
- [Reward Constrained Interactive Recommendation with Natural Language Feedback](https://mlsanity.com/paper/2005.01618/) (2020)
- [Fashion-IQ 2020 Challenge 2nd Place Team's Solution](https://mlsanity.com/paper/2007.06404/) (2020)

## Built on by (most cited first, 29 of 29)

- [LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant](https://mlsanity.com/paper/2412.01720/) (2024)
- [Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features](https://mlsanity.com/paper/2308.11485/) (2023)
- [Diffusion Models for Generative Outfit Recommendation](https://mlsanity.com/paper/2402.17279/) (2024)
- [Progressive Learning for Image Retrieval with Hybrid-Modality Queries](https://mlsanity.com/paper/2204.11212/) (2022)
- [Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2503.19296/) (2025)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [FashionNTM: Multi-turn Fashion Image Retrieval via Cascaded Memory](https://mlsanity.com/paper/2308.10170/) (2023)
- [U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMs](https://mlsanity.com/paper/2507.14902/) (2025)
- [Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search](https://mlsanity.com/paper/2402.07742/) (2024)
- [FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning](https://mlsanity.com/paper/2210.15028/) (2022)
- [UIGR: Unified Interactive Garment Retrieval](https://mlsanity.com/paper/2204.03111/) (2022)
- [ChatSearch: a Dataset and a Generative Retrieval Model for General Conversational Image Retrieval](https://mlsanity.com/paper/2410.18715/) (2024)
- [Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval](https://mlsanity.com/paper/2501.15379/) (2025)
- [Training and challenging models for text-guided fashion image retrieval](https://mlsanity.com/paper/2204.11004/) (2022)
- [Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information](https://mlsanity.com/paper/2305.01788/) (2023)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [Query Understanding in LLM-based Conversational Information Seeking](https://mlsanity.com/paper/2504.06356/) (2025)
- [LRVS-Fashion: Extending Visual Search with Referring Instructions](https://mlsanity.com/paper/2306.02928/) (2023)
- [Sentiment Analysis of Fashion Related Posts in Social Media](https://mlsanity.com/paper/2111.07815/) (2021)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval](https://mlsanity.com/paper/2602.06034/) (2026)
- [ReCQR: Incorporating conversational query rewriting to improve Multimodal Image Retrieval](https://mlsanity.com/paper/2603.26669/) (2026)
- [UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrieval](https://mlsanity.com/paper/2604.20318/) (2026)
- [Social Media Fashion Knowledge Extraction as Captioning](https://mlsanity.com/paper/2309.16270/) (2023)
- [Test-Time Scaling Strategies for Generative Retrieval in Multimodal Conversational Recommendations](https://mlsanity.com/paper/2508.18132/) (2025)
- [Interactive Garment Recommendation with User in the Loop](https://mlsanity.com/paper/2402.11627/) (2024)
- [Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering](https://mlsanity.com/paper/2409.16025/) (2024)
- [CIRCLED: A Multi-turn CIR Dataset with Consistent Dialogues across Domains](https://mlsanity.com/paper/2605.26734/) (2026)
- [Diverse-Intent Multi-Turn Fashion Image Retrieval](https://mlsanity.com/paper/2607.20291/) (2026)

## Abstract (arXiv)

We study the task of conversational fashion image retrieval via multiturn natural language feedback. Most previous studies are based on single-turn settings. Existing models on multiturn conversational fashion image retrieval have limitations, such as employing traditional models, and leading to ineffective performance. We propose a novel framework that can effectively handle conversational fashion image retrieval with multiturn natural language feedback texts. One characteristic of the framework is that it searches for candidate images based on exploitation of the encoded reference image and feedback text information together with the conversation history. Furthermore, the image fashion attribute information is leveraged via a mutual attention strategy. Since there is no existing fashion dataset suitable for the multiturn setting of our task, we derive a large-scale multiturn fashion dataset via additional manual annotation efforts on an existing single-turn dataset. The experiments show that our proposed model significantly outperforms existing state-of-the-art methods.