mlsanityData to 7 Oct 2026

Shopping Companion: Benchmarking and Training LLM Agents for Long-Horizon Preference-Grounded E-Commerce Tasks

arXiv 2603.14864 ↗First posted 16 Mar 2026cs.CLcited by 5

On the map: LLM agents and tool use → Social simulation and multi-agent debate

also in 1 more
Abstract (arXiv)

In e-commerce, LLM agents show promise for shopping tasks such as recommendations, budget management, and bundle deals, where accurately capturing user preferences from long-horizon conversations is critical. However, progress is limited by two key challenges: (1) the absence of benchmarks for evaluating long-term preference-aware shopping tasks, and (2) the lack of fine-grained supervision for shopping agent training. To fill the benchmark gap, we introduce Shopping Companion Bench, a novel benchmark comprising two shopping tasks that require cross-session preference memory, grounded in a product pool of over 1.2 million real-world items. Our analysis further identifies two major sources of failure on this benchmark: cascading errors caused by preference hallucination, and insufficient verification of product attributes against user requirements. To address these failure modes, we design annotation-free, tool-wise rewards that provide process supervision for each tool call, alleviating reward sparsity in long-horizon tasks. Experimental results demonstrate that even state-of-the-art models such as GPT-5 achieve success rates below 70%, highlighting the difficulty of our benchmark. Notably, our fine-tuned lightweight 4B model consistently outperforms strong baselines in both preference capture and task performance, suggesting the effectiveness of our reward design.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 14 of its 31 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

14 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents6 Aug

Other references · 13

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  6. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  7. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  8. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  9. 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 372
  10. 2021Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representationscited by 60
  11. 2019Conversational Product Search Based on Negative Feedbackcited by 13
  12. 2025Wizard of Shopping: Target-Oriented E-commerce Dialogue Generation with Decision Tree Branchingcited by 10
1 more
  1. 2024Learning to Ask: Conversational Product Search via Representation Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent
  2. 2026Customer-Agent: Overcoming Context Limitations in Ultra-Long Shopping Trajectories via Tool-Augmented Agents and RLVR
  3. 2026ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants
  4. 2026E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation
  5. 2026MemRerank: Preference Memory for Personalized Product Reranking
  6. 2026RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce
  7. 2026MerchantBench: Benchmarking LLM Agents for Long-Term Coherence in E-Commerce Operations
  8. 2026RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents