mlsanityData to 14 Oct 2026

BASES: Large-scale Web Search User Simulation with Large Language Model based Agents

arXiv 2402.17505 ↗First posted 27 Feb 2024cs.IR cs.CLcited by 25

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation. Considering the scarcity and limit (e.g., privacy issues) of real user data, in this paper, we conduct large-scale user simulation for web search, to improve the analysis and modeling of user search behavior. Specially, we propose BASES, a novel user simulation framework with LLM-based agents, designed to facilitate comprehensive simulations of web search user behaviors. Our simulation framework can generate unique user profiles at scale, which subsequently leads to diverse search behaviors. To demonstrate the effectiveness of BASES, we conduct evaluation experiments based on two human benchmarks in both Chinese and English, demonstrating that BASES can effectively simulate large-scale human-like search behaviors. To further accommodate the research on web search, we develop WARRIORS, a new large-scale dataset encompassing web search user behaviors, including both Chinese and English versions, which can greatly bolster research in the field of information retrieval. Our code and data will be publicly released soon.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 52 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

16 of 52 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  6. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  7. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  8. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 176
  9. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 147
  10. 2022Dense Text Retrieval based on Pretrained Language Models: A Surveycited by 136
  11. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
  12. 2020ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Searchcited by 34
4 more
  1. 2023TOME: A Two-stage Approach for Model-based Retrievalcited by 21
  2. 2021Contrastive Learning of User Behavior Sequence for Context-Aware Document Rankingcited by 18
  3. 2018A Click Sequence Model for Web Searchcited by 11
  4. 2021Group based Personalized Search by Integrating Search Behaviour and Friend Networkcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 10; 2026: 11.

25 papers.

Sort:

By area:

  1. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 82
  2. 2026Mind the Sim2Real Gap in User Simulation for Agentic Taskscited by 54
  3. 2025ChatBench: From Static Benchmarks to Human-AI Evaluationcited by 34
  4. 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulationscited by 28
  5. 2025A Survey of Personalization: From RAG to Agentcited by 27
  6. 2025Towards Agentic Recommender Systems in the Era of Multimodal Large Language Modelscited by 26
  7. 2024Very Large-Scale Multi-Agent Simulation in AgentScopecited by 18
  8. 2026A Survey on LLM-based Conversational User Simulationcited by 18
  9. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigmscited by 17
  10. 2024Social Skill Training with Large Language Modelscited by 15
  11. 2026RealUserSim: Bridging the Reality Gap in Agent Benchmarking via Grounded User Simulationcited by 12
  12. 2026Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Tracescited by 9
13 more
  1. 2025Modeling Earth-Scale Human-Like Societies with One Billion Agentscited by 8
  2. 2026Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Auditscited by 8
  3. 2024AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMscited by 7
  4. 2025Query Understanding in LLM-based Conversational Information Seekingcited by 5
  5. 2026Reinforcing Human Behavior Simulation via Verbal Feedbackcited by 4
  6. 2025Crucible: Quantifying the Potential of Control Algorithms through LLM Agentscited by 1
  7. 2025REVISION:Reflective Intent Mining and Online Reasoning Auxiliary for E-commerce Visual Search System Optimizationcited by 1
  8. 2025Artificial Intelligence and Civil Discourse: How LLMs Moderate Climate Change Conversationscited by 0
  9. 2026Story2Proposal: A Scaffold for Structured Scientific Paper Writingcited by 0
  10. 2026Computational KJ-Ho: An Analyst-Bias-Free Insight Extraction Framework from Large-Scale Qualitative Data Using Domain-Specialized LLMscited by 0
  11. 2026The "Curse of Knowledge" in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusioncited by 0
  12. 2026Are LLMs Good Financial User Simulators? Multi-view Investor Logic Alignment (MILA)cited by 0
  13. 2026PsyMemBehav: Psychology-Informed and Memory-Grounded Two-Level Identity-to-Action Reasoning for Social-Media Behavior Simulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023User Behavior Simulation with Large Language Model based Agents
  2. 2024Level-Navi Agent: A Framework and benchmark for Chinese Web Search Agents
  3. 2025WideSearch: Benchmarking Agentic Broad Info-Seeking
  4. 2025Exploring Human-Like Thinking in Search Simulations with Large Language Models
  5. 2026Verifiable User Simulation for Search and Recommendation Systems
  6. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges
  7. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
  8. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval