Similar Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents

arXiv 2607.10198 ↗First posted 11 Jul 2026cs.CLcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Search APIs expose ranked snippets, URLs, and metadata on which agents decide whether to answer, search again, or fetch pages. We evaluate these interfaces as decision surfaces on a fixed sample of 100 questions from the 254-question SealQA-Hard subset, using one frozen GPT-5.4 agent, a fixed orchestration harness, and a shared page-fetch backend across Brave, Tavily, and Firecrawl. A Kimi-K2.6 oracle labels visible URL-level evidence; a separate answer audit yields 25, 25, and 26 correct answers out of 100. These counts indicate similar observed accuracy but do not establish equivalence. Under the tested configurations, Brave exposes more pre-fetch support alongside a larger snippet surface; Tavily has a larger rank-1 share among trajectory-pooled supporting observations; and Firecrawl is associated with broader exploration. First-search query-level metrics distinguish support availability from ranking, while contradiction exposure complements contradiction-to-gold ratios. A retrospective oracle union covers 44/100 questions, versus 26/100 for the best individual provider: 18 percentage points of headroom. The observed evidence and action differences motivate evaluating search APIs jointly with agent policy and retrieval budget.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 31 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

23 of 31 references.

Closest ancestors · 1

Same direction, newest first

2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul

Other references · 22

2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210

10 more

2023GAIA: a benchmark for General AI Assistantscited by 1,128
2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
2023Evaluating Verifiability in Generative Search Enginescited by 303
2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 265
2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 75
2026Over-Searching in Search-Augmented Large Language Modelscited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way