# Characterizing Web Search by Conversational LLM Agents: From Search Decisions and Strategies to Results and Responses

arXiv: https://arxiv.org/abs/2609.19244 · HTML: https://mlsanity.com/paper/2609.19244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 18 of 18)

- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2503.05592/) (2025)
- [WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents](https://mlsanity.com/paper/2509.06501/) (2025)
- [Deep Research Bench: Evaluating AI Web Research Agents](https://mlsanity.com/paper/2506.06287/) (2025)
- [InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.15872/) (2025)
- [ShareChat: A Dataset of Chatbot Conversations in the Wild](https://mlsanity.com/paper/2512.17843/) (2025)
- [Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents](https://mlsanity.com/paper/2605.06635/) (2026)
- [VeriWeb: Verifiable Long-Chain Web Benchmark for Agentic Information-Seeking](https://mlsanity.com/paper/2508.04026/) (2025)
- [To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling](https://mlsanity.com/paper/2605.00737/) (2026)
- [Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents](https://mlsanity.com/paper/2604.03173/) (2026)
- [Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs](https://mlsanity.com/paper/2511.18931/) (2025)
- [How You Ask Matters! Adaptive RAG Robustness to Query Variations](https://mlsanity.com/paper/2604.10745/) (2026)
- [Setting the Course, but Forgetting to Steer: Analyzing Compliance with GDPR's Right of Access to Data by Instagram, TikTok, and YouTube](https://mlsanity.com/paper/2502.11208/) (2025)
- [A Picture of Agentic Search](https://mlsanity.com/paper/2602.17518/) (2026)

## Abstract (arXiv)

Conversational LLM agents increasingly rely on Web search, yet the end-to-end lifecycle of agentic search remains poorly understood. We present the first study of Web search across four major conversational platforms (ChatGPT, Claude, Grok, and DeepSeek), combining real-world user interactions (invivo) with controlled experiments using the same platform's models by their APIs (invitro). We investigate the quality of agentic decisions to invoke Web search, their strategies to formulate queries, the potential domain preferences in the search results they receive, and the choices they make when transforming search results into grounded responses. We find that Web-search decisions vary substantially across platforms and models, while more frequent Web-search invocation does not necessarily yield better response quality. We further show that conversational agents employ different complex querying strategies and that platform specific search engines return search results from their preferred domains. Finally, although responses are largely grounded in search results, some claims rely on uncited search results, raising concerns about attribution and reliability. Our findings have important implications for the design of future AI agents and Web search tools optimized for conversational retrieval.