mlsanityData to 7 Oct 2026

Imagery as Inquiry: Exploring A Multimodal Dataset for Conversational Recommendation

arXiv 2405.14142 ↗First posted 23 May 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a multimodal dataset where users express preferences through images. These images encompass a broad spectrum of visual expressions ranging from landscapes to artistic depictions. Users request recommendations for books or music that evoke similar feelings to those captured in the images, and recommendations are endorsed by the community through upvotes. This dataset supports two recommendation tasks: title generation and multiple-choice selection. Our experiments with large foundation models reveal their limitations in these tasks. Particularly, vision-language models show no significant advantage over language-only counterparts that use descriptions, which we hypothesize is due to underutilized visual capabilities. To better harness these abilities, we propose the chain-of-imagery prompting, which results in notable improvements. We release our code and datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

19 of 34 references.

References · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  4. 2015Image-based Recommendations on Styles and Substitutescited by 702
  5. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  6. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
  7. 2017Objects that Soundcited by 254
  8. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  9. 2015VBPR: Visual Bayesian Personalized Ranking from Implicit Feedbackcited by 151
  10. 2023Large Language Models as Zero-Shot Conversational Recommenderscited by 126
  11. 2020Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusioncited by 122
  12. 2019Towards Knowledge-Based Recommender Dialog Systemcited by 112
7 more
  1. 2021VLM: Task-agnostic Video-Language Model Pre-training for Video Understandingcited by 102
  2. 2018Dialog-based Interactive Image Retrievalcited by 84
  3. 2022User-Centric Conversational Recommendation with Multi-Aspect User Modelingcited by 29
  4. 2024Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyondcited by 25
  5. 2016Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendationcited by 20
  6. 2023Language-Guided Music Recommendation for Video via Prompt Analogiescited by 18
  7. 2021On component interactions in two-stage recommender systemscited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models
  2. 2026VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models
  3. 2025LaViC: Adapting Large Vision-Language Models to Visually-Aware Conversational Recommendation
  4. 2025VOGUE: A Multimodal Dataset for Conversational Recommendation in Fashion
  5. 2025Do Recommender Systems Really Leverage Multimodal Content? A Comprehensive Analysis on Multimodal Representations for Recommendation
  6. 2026OpticalRec: Unified Optical Vision-Language Representation for Multimodal Recommendation
  7. 2024X-Reflect: Cross-Reflection Prompting for Multimodal Recommendation
  8. 2025Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation