mlsanityData to 7 Oct 2026

ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions

arXiv 2407.00942 ↗First posted 1 Jul 2024cs.IR cs.AI cs.CLcited by 10

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

This paper introduces the task of product demand clarification within an e-commercial scenario, where the user commences the conversation with ambiguous queries and the task-oriented agent is designed to achieve more accurate and tailored product searching by asking clarification questions. To address this task, we propose ProductAgent, a conversational information seeking agent equipped with abilities of strategic clarification question generation and dynamic product retrieval. Specifically, we develop the agent with strategies for product feature summarization, query generation, and product retrieval. Furthermore, we propose the benchmark called PROCLARE to evaluate the agent's performance both automatically and qualitatively with the aid of a LLM-driven user simulator. Experiments show that ProductAgent interacts positively with the user and enhances retrieval performance with increasing dialogue turns, where user demands become gradually more explicit and detailed. All the source codes will be released after the review anonymity period.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 93 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

39 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul

Other references · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  11. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  12. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
26 more
  1. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  2. 2021Alignment of Language Agentscited by 167
  3. 2021Improving Named Entity Recognition by External Context Retrieving and Cooperative Learningcited by 76
  4. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  5. 2022CLAM: Selective Clarification for Ambiguous Questions with Generative Language Modelscited by 66
  6. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  7. 2023On the (In)Effectiveness of Large Language Models for Chinese Text Correctioncited by 53
  8. 2024Evaluating Large Language Models as Generative User Simulators for Conversational Recommendationcited by 45
  9. 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 44
  10. 2023LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzlescited by 41
  11. 2023A Survey on Asking Clarification Questions Datasets in Conversational Systemscited by 28
  12. 2024QueryAgent: A Reliable and Efficient Reasoning Framework with Environmental Feedback-based Self-Correctioncited by 27
  13. 2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 26
  14. 2022Contrastive Learning with Hard Negative Entities for Entity Set Expansioncited by 26
  15. 2024Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmarkcited by 22
  16. 2023In-Context Learning User Simulators for Task-Oriented Dialog Systemscited by 19
  17. 2023SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understandingcited by 18
  18. 2020AliMe KG: Domain Knowledge Graph Construction and Application in E-commercecited by 16
  19. 2023A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Checkcited by 15
  20. 2023MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entitiescited by 13
  21. 2024Evaluating Robustness of Generative Search Engine on Adversarial Factual Questionscited by 9
  22. 2024Question Suggestion for Conversational Shopping Assistants Using Product Metadatacited by 8
  23. 2023Towards Asking Clarification Questions for Information Seeking on Task-Oriented Dialoguescited by 7
  24. 2023Large Language Models Should Ask Clarifying Questions to Increase Confidence in Generated Codecited by 4
  25. 2023CIRCLE: Multi-Turn Query Clarifications with Reinforcement Learningcited by 4
  26. 2018Query Tracking for E-commerce Conversational Search: A Machine Comprehension Perspectivecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2025LLM Agents for Education: Advances and Applicationscited by 59
  2. 2026RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulationcited by 8
  3. 2026EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configurationcited by 7
  4. 2025Towards Personalized Conversational Sales Agents: Contextual User Profiling for Strategic Actioncited by 6
  5. 2026IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agentscited by 3
  6. 2025PSCon: Product Search Through Conversationscited by 1
  7. 2026Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoningcited by 1
  8. 2025FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writingcited by 0
  9. 2026From Atomic to Agentic: Towards Interpretable Evaluation of LLMs' Agentic Mathematical Capabilitiescited by 0
  10. 2026From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scalecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs
  2. 2026A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models
  3. 2026QueryAgent-R1: Bridging Query Generation and Product Retrieval for E-Commerce Query Recommendation
  4. 2026Characterizing Web Search by Conversational LLM Agents: From Search Decisions and Strategies to Results and Responses
  5. 2025Contextually Aware E-Commerce Product Question Answering using RAG
  6. 2026CIR at iKAT SCAI 2026: Exploring Clarification Need Prediction in Agentic Conversational Search
  7. 2024STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents
  8. 2024Identifying Shopping Intent in Product QA for Proactive Recommendations