mlsanityData to 14 Oct 2026

VisualLens: Personalization through Task-Agnostic Visual History

arXiv 2411.16034 ↗First posted 25 Nov 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing recommendation systems either rely on user interaction logs, such as online shopping history for shopping recommendations, or focus on text signals. However, item-based histories are not always accessible, and are not generalizable for multimodal recommendation. We hypothesize that a user's visual history -- comprising images from daily life -- can offer rich, task-agnostic insights into their interests and preferences, and thus be leveraged for effective personalization. To this end, we propose VisualLens, a novel framework that leverages multimodal large language models (MLLMs) to enable personalization using task-agnostic visual history. VisualLens extracts, filters, and refines a spectrum user profile from the visual history to support personalized recommendation. We created two new benchmarks, Google-Review-V and Yelp-V, with task-agnostic visual histories, and show that VisualLens improves over state-of-the-art item-based multimodal recommendations by 5-10% on Hit@3, and outperforms GPT-4o by 2-5%. Further analysis shows that VisualLens is robust across varying history lengths and excels at adapting to both longer histories and unseen content categories.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

40 of 59 references.

References · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  3. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  4. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 476
  5. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  6. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
  7. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 309
  8. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 280
  9. 2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 172
  10. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  11. 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 161
  12. 2015VBPR: Visual Bayesian Personalized Ranking from Implicit Feedbackcited by 151
28 more
  1. 2023Text Is All You Need: Learning Language Representations for Sequential Recommendationcited by 149
  2. 2022Towards Universal Sequence Representation Learning for Recommender Systemscited by 121
  3. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
  4. 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 77
  5. 2023Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferencescited by 66
  6. 2023Collaborative Large Language Model for Recommender Systemscited by 64
  7. 2023GenRec: Large Language Model for Generative Recommendationcited by 63
  8. 2023ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Modelscited by 54
  9. 2023Teach LLMs to Personalize -- An Approach inspired by Writing Educationcited by 48
  10. 2023Multi-Modal Self-Supervised Learning for Recommendationcited by 45
  11. 2022Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendationcited by 42
  12. 2024IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representationscited by 38
  13. 2022UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Miningcited by 33
  14. 2022Personalized Showcases: Generating Multi-Modal Explanations for Recommendationscited by 30
  15. 2018Adversarial Training Towards Robust Multimedia Recommender Systemcited by 28
  16. 2022TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedbackcited by 27
  17. 2024Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyondcited by 25
  18. 2024Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?cited by 24
  19. 2024Enhancing Taobao Display Advertising with Multimodal Representations: Challenges, Approaches and Insightscited by 23
  20. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  21. 2024TLDR: Token-Level Detective Reward Model for Large Vision Language Modelscited by 15
  22. 2023Prompt Tuning Large Language Models on Personalized Aspect Extraction for Recommendationscited by 13
  23. 2023LKPNR: LLM and KG for Personalized News Recommendation Frameworkcited by 13
  24. 2022Generalization Differences between End-to-End and Neuro-Symbolic Vision-Language Reasoning Systemscited by 6
  25. 2024Enhancing Content-based Recommendation via Large Language Modelcited by 6
  26. 2023Efficient End-to-End Visual Document Understanding with Rationale Distillationcited by 5
  27. 2023COURIER: Contrastive User Intention Reconstruction for Large-Scale Visual Recommendationcited by 2
  28. 2024A Model-based Multi-Agent Personalized Short-Video Recommender Systemcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models
  2. 2025Bridging Collaborative Filtering and Large Language Models with Dynamic Alignment, Multimodal Fusion and Evidence-grounded Explanations
  3. 2024Multimodal Pretraining, Adaptation, and Generation for Recommendation: A Survey
  4. 2024Imagery as Inquiry: Exploring A Multimodal Dataset for Conversational Recommendation
  5. 2026VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models
  6. 2025LaViC: Adapting Large Vision-Language Models to Visually-Aware Conversational Recommendation
  7. 2025Serendipitous Recommendation with Multimodal LLM
  8. 2025Generate, Not Recommend: Personalized Multimodal Content Generation