# AgenticShop: Benchmarking Agentic Product Curation for Personalized Web Shopping

arXiv: https://arxiv.org/abs/2602.12315 · HTML: https://mlsanity.com/paper/2602.12315/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 31 of its 62 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 30 of 31)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [An Illusion of Progress? Assessing the Current State of Web Agents](https://mlsanity.com/paper/2504.01382/) (2025)
- [Multimodal Web Navigation with Instruction-Finetuned Foundation Models](https://mlsanity.com/paper/2305.11854/) (2023)
- [Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents](https://mlsanity.com/paper/2411.06559/) (2024)
- [WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences](https://mlsanity.com/paper/2306.07906/) (2023)
- [Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems](https://mlsanity.com/paper/2407.13032/) (2024)
- [Beyond Browsing: API-Based Web Agents](https://mlsanity.com/paper/2410.16464/) (2024)
- [Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge](https://mlsanity.com/paper/2506.21506/) (2025)
- [Large Language Models Empowered Personalized Web Agents](https://mlsanity.com/paper/2410.17236/) (2024)
- [MMInA: Benchmarking Multihop Multimodal Internet Agents](https://mlsanity.com/paper/2404.09992/) (2024)
- [DeepShop: A Benchmark for Deep Research Shopping Agents](https://mlsanity.com/paper/2506.02839/) (2025)
- [Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generation](https://mlsanity.com/paper/2307.09688/) (2023)
- [Search Arena: Analyzing Search-Augmented LLMs](https://mlsanity.com/paper/2506.05334/) (2025)
- [ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents](https://mlsanity.com/paper/2508.04266/) (2025)
- [Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models](https://mlsanity.com/paper/2410.20745/) (2024)
- [WebMall -- A Multi-Shop Benchmark for Evaluating Web Agents](https://mlsanity.com/paper/2508.13024/) (2025)
- [Learning Latent Vector Spaces for Product Search](https://mlsanity.com/paper/1608.07253/) (2016)
- [Neural IR Meets Graph Embedding: A Ranking Model for Product Search](https://mlsanity.com/paper/1901.08286/) (2019)
- [ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2502.20196/) (2025)
- [A Functionality-Grounded Benchmark for Evaluating Web Agents in E-commerce Domains](https://mlsanity.com/paper/2508.15832/) (2025)
- [BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback](https://mlsanity.com/paper/2509.21106/) (2025)
- [Optimizing E-commerce Search: Toward a Generalizable and Rank-Consistent Pre-Ranking Model](https://mlsanity.com/paper/2405.05606/) (2024)

## Built on by (most cited first, 7 of 7)

- [RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems](https://mlsanity.com/paper/2605.11874/) (2026)
- [AlphaEval: Evaluating Agents in Production](https://mlsanity.com/paper/2604.12162/) (2026)
- [Valley3: Scaling Omni Foundation Models for E-commerce](https://mlsanity.com/paper/2605.01278/) (2026)
- [A Decision-Centered Reference Architecture for Trustworthy Agentic Commerce](https://mlsanity.com/paper/2607.18347/) (2026)
- [WorldBench: Culturally Grounded Benchmark for Multilingual Agents](https://mlsanity.com/paper/2609.01056/) (2026)
- [UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs](https://mlsanity.com/paper/2608.10042/) (2026)
- [Webhell: A Unified Interface for Agentic Web Access](https://mlsanity.com/paper/2610.13416/) (2026)

## Abstract (arXiv)

The proliferation of e-commerce has made web shopping platforms key gateways for customers navigating the vast digital marketplace. Yet this rapid expansion has led to a noisy and fragmented information environment, increasing cognitive burden as shoppers explore and purchase products online. With promising potential to alleviate this challenge, agentic systems have garnered growing attention for automating user-side tasks in web shopping. Despite significant advancements, existing benchmarks fail to comprehensively evaluate how well agentic systems can curate products in open-web settings. Specifically, they have limited coverage of shopping scenarios, focusing only on simplified single-platform lookups rather than exploratory search. Moreover, they overlook personalization in evaluation, leaving unclear whether agents can adapt to diverse user preferences in realistic shopping contexts. To address this gap, we present AgenticShop, the first benchmark for evaluating agentic systems on personalized product curation in open-web environment. Crucially, our approach features realistic shopping scenarios, diverse user profiles, and a verifiable, checklist-driven personalization evaluation framework. Through extensive experiments, we demonstrate that current agentic systems remain largely insufficient, emphasizing the need for user-side systems that effectively curate tailored products across the modern web.