mlsanityData to 14 Oct 2026

Towards Agentic Recommender Systems in the Era of Multimodal Large Language Models

arXiv 2503.16734 ↗First posted 20 Mar 2025cs.AI cs.IRcited by 26

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

also in 1 more
Abstract (arXiv)

Recent breakthroughs in Large Language Models (LLMs) have led to the emergence of agentic AI systems that extend beyond the capabilities of standalone models. By empowering LLMs to perceive external environments, integrate multimodal information, and interact with various tools, these agentic systems exhibit greater autonomy and adaptability across complex tasks. This evolution brings new opportunities to recommender systems (RS): LLM-based Agentic RS (LLM-ARS) can offer more interactive, context-aware, and proactive recommendations, potentially reshaping the user experience and broadening the application scope of RS. Despite promising early results, fundamental challenges remain, including how to effectively incorporate external knowledge, balance autonomy with controllability, and evaluate performance in dynamic, multimodal settings. In this perspective paper, we first present a systematic analysis of LLM-ARS: (1) clarifying core concepts and architectures; (2) highlighting how agentic capabilities -- such as planning, memory, and multimodal reasoning -- can enhance recommendation quality; and (3) outlining key research questions in areas such as safety, efficiency, and lifelong personalization. We also discuss open problems and future directions, arguing that LLM-ARS will drive the next wave of RS innovation. Ultimately, we foresee a paradigm shift toward intelligent, autonomous, and collaborative recommendation experiences that more closely align with users' evolving needs and complex decision-making processes.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 81 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

42 of 81 references.

References · 42

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  5. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  6. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
  7. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  8. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
  9. 2024Understanding the planning of LLM agents: A surveycited by 421
  10. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 174
  11. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 170
  12. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 145
30 more
  1. 2023Recommender AI Agent: Integrating Large Language Models for Interactive Recommendationscited by 131
  2. 2023RecMind: Large Language Model Powered Agent For Recommendationcited by 129
  3. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Datacited by 126
  4. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
  5. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 112
  6. 2019Reinforcement Knowledge Graph Reasoning for Explainable Recommendationcited by 102
  7. 2019RecSim: A Configurable Simulation Platform for Recommender Systemscited by 94
  8. 2022Personalized Prompt Learning for Explainable Recommendationcited by 79
  9. 2024User-LLM: Efficient LLM Contextualization with User Embeddingscited by 76
  10. 2024Large Language Models Empowered Personalized Web Agentscited by 66
  11. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  12. 2024AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent Systemcited by 53
  13. 2024Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learningcited by 51
  14. 2024CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendationcited by 44
  15. 2024A Multi-Agent Conversational Recommender Systemcited by 43
  16. 2024LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendationscited by 40
  17. 2024USimAgent: Large Language Models for Simulating Search Userscited by 32
  18. 2024AI PERSONA: Towards Life-long Personalization of LLMscited by 32
  19. 2023RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agentscited by 30
  20. 2024BASES: Large-scale Web Search User Simulation with Large Language Model based Agentscited by 25
  21. 2021Generate Natural Language Explanations for Recommendationcited by 23
  22. 2024RDRec: Rationale Distillation for LLM-based Recommendationcited by 22
  23. 2024Prospect Personalized Recommendation on Large Language Model-based Agent Platformcited by 19
  24. 2023Automated Interactive Domain-Specific Conversational Agents that Understand Human Dialogscited by 12
  25. 2024SUBER: An RL Environment with Simulated Human Behavior for Recommender Systemscited by 11
  26. 2024Enhancing High-order Interaction Awareness in LLM-based Recommender Modelcited by 8
  27. 2024Reinforcement Learning-based Recommender Systems with Large Language Models for State Reward and Action Modelingcited by 7
  28. 2024ULMRec: User-centric Large Language Model for Sequential Recommendationcited by 4
  29. 2025AgentRec: Agent Recommendation Using Sentence Embeddings Aligned to Human Feedbackcited by 3
  30. 2025Large Language Model driven Policy Exploration for Recommender Systemscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 15.

26 papers.

Sort:

By area:

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 42
  3. 2025AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systemscited by 23
  4. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  5. 2025RecoWorld: Building Simulated Environments for Agentic Recommender Systemscited by 19
  6. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigmscited by 17
  7. 2026Skill-R1: Agent Skill Evolution via Reinforcement Learningcited by 15
  8. 2026Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editablecited by 13
  9. 2025DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agentscited by 12
  10. 2025Pluralistic Off-policy Evaluation and Alignmentcited by 11
  11. 2025WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoningcited by 10
  12. 2025In-context Ranking Preference Optimizationcited by 9
14 more
  1. 2026AMPS: Adaptive Modality Preference Steering via Functional Entropycited by 4
  2. 2026AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Webcited by 4
  3. 2026Autonomous Information Seeking: A Roadmap for Agentic Recommender Systemscited by 4
  4. 2026Factorized Latent Reasoning for LLM-based Recommendationcited by 3
  5. 2026Verifiable Reasoning for LLM-based Generative Recommendationcited by 2
  6. 2026OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agentscited by 2
  7. 2026Skill-CMIB: Multimodal Agent Skill for Consistent Action via Conditional Multimodal Information Bottleneckcited by 1
  8. 2026F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Rankingcited by 1
  9. 2026KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMscited by 1
  10. 2025To See or To Read: User Behavior Reasoning in Multimodal LLMscited by 0
  11. 2026AgenticRec: A Recommendation-Oriented Agentic Framework with Progressive Tool-Integrated Reasoning Optimizationcited by 0
  12. 2026Fair Agents: Balancing Multistakeholder Alignment in Multi-Agent Personalization Systemscited by 0
  13. 2026RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agentscited by 0
  14. 2026Personal-Agent Mediated Recommendation with Cross-Platform User Historycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on LLM-powered Agents for Recommender Systems
  2. 2025The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems
  3. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval
  4. 2026Shape Your Feed: An LLM-based Agentic System for Conversational Recommendation
  5. 2026Personalized Recommendation Tool Learning via Autonomous Language Agents
  6. 2024All Roads Lead to Rome: Unveiling the Trajectory of Recommender Systems Across the LLM Era
  7. 2026AMEM4Rec: Leveraging Cross-User Similarity for Memory Evolution in Agentic LLM Recommenders
  8. 2026Seeing and Reflecting: Multimodal Memory-Enhanced Agent Collaboration for Recommendation