Shopping Companion: Benchmarking and Training LLM Agents for Long-Horizon Preference-Grounded E-Commerce Tasks
On the map: LLM agents and tool use → Social simulation and multi-agent debate
also in 1 more
Abstract (arXiv)
In e-commerce, LLM agents show promise for shopping tasks such as recommendations, budget management, and bundle deals, where accurately capturing user preferences from long-horizon conversations is critical. However, progress is limited by two key challenges: (1) the absence of benchmarks for evaluating long-term preference-aware shopping tasks, and (2) the lack of fine-grained supervision for shopping agent training. To fill the benchmark gap, we introduce Shopping Companion Bench, a novel benchmark comprising two shopping tasks that require cross-session preference memory, grounded in a product pool of over 1.2 million real-world items. Our analysis further identifies two major sources of failure on this benchmark: cascading errors caused by preference hallucination, and insufficient verification of product attributes against user requirements. To address these failure modes, we design annotation-free, tool-wise rewards that provide process supervision for each tool call, alleviating reward sparsity in long-horizon tasks. Experimental results demonstrate that even state-of-the-art models such as GPT-5 achieve success rates below 70%, highlighting the difficulty of our benchmark. Notably, our fine-tuned lightweight 4B model consistently outperforms strong baselines in both preference capture and task performance, suggesting the effectiveness of our reward design.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 14 of its 31 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
14 of 31 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 372
- 2021Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representationscited by 60
- 2019Conversational Product Search Based on Negative Feedbackcited by 13
- 2025Wizard of Shopping: Target-Oriented E-commerce Dialogue Generation with Decision Tree Branchingcited by 10
1 more
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agentscited by 4
- 2026ComboShoppingBench: Evaluating LLM Agents for Budget-Constrained Basket Shopping with Couponscited by 2
- 2026Paying to Know: Micro-Transaction Markets for Verified Product Information in Agentic E-Commercecited by 0
- 2026MMShopBench: A Real-Log Benchmark for Multimodal, Multi-Turn Shopping Agentscited by 0
- 2026PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agentscited by 0
- 2026PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agents28 Sep 2026
- 2026ComboShoppingBench: Evaluating LLM Agents for Budget-Constrained Basket Shopping with Coupons10 Aug 2026
- 2026MMShopBench: A Real-Log Benchmark for Multimodal, Multi-Turn Shopping Agents31 Jul 2026
- 2026Paying to Know: Micro-Transaction Markets for Verified Product Information in Agentic E-Commerce23 Jun 2026
- 2026ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents15 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent
- 2026Customer-Agent: Overcoming Context Limitations in Ultra-Long Shopping Trajectories via Tool-Augmented Agents and RLVR
- 2026ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants
- 2026E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation
- 2026MemRerank: Preference Memory for Personalized Product Reranking
- 2026RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce
- 2026MerchantBench: Benchmarking LLM Agents for Long-Term Coherence in E-Commerce Operations
- 2026RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents