mlsanityData to 7 Oct 2026

When Should LLMs Search? Counterfactual Supervision for Search Routing

arXiv 2607.05752 ↗First posted 7 Jul 2026cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Search-augmented language models can use external evidence to compensate for limitations in parametric knowledge, but search is not uniformly beneficial: models may call search for questions they can already answer, or rely on noisy evidence when correction, clarification, or abstention would be more appropriate. We formulate this as an instance-level search-routing problem: deciding whether search is needed to improve task success relative to a no-search execution. To derive supervision, we compare no-search and forced-search outcomes for the same question and construct an oracle over NO SEARCH, SEARCH, and UNSOLVED based on task-specific success. Using this oracle as both an evaluation criterion and a learning signal, we train search-routing policies with supervised fine-tuning and preference optimization, improving routing macro-F1 on oracle-eligible examples from 0.7082 to 0.8235 for Gemma E2B and from 0.7053 to 0.8365 for Qwen3.5-4B. Further analysis shows that the learned policies reduce model-specific routing failures: Gemma primarily learns no-search restraint, while Qwen further reduces missed search; residual UNSOLVED cases reveal heterogeneous bottlenecks involving model capacity, retrieval budget, evidence use, and policy behavior.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 28 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

23 of 28 references.

References · 23

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  10. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  11. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  12. 2023Active Retrieval Augmented Generationcited by 506
11 more
  1. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  2. 2024Measuring short-form factuality in large language modelscited by 345
  3. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
  4. 2024Iterative Reasoning Preference Optimizationcited by 244
  5. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  6. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
  7. 2023Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Modelscited by 60
  8. 2024Can AI Assistants Know What They Don't Know?cited by 47
  9. 2025When2Call: When (not) to Call Toolscited by 36
  10. 2024ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Modelscited by 25
  11. 2026Over-Searching in Search-Augmented Large Language Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnessescited by 1
  2. 2026What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
  2. 2026LinTree: Improving LLM Reasoning with Explicitly Structured Search Histories
  3. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
  4. 2025Rethinking the Unsolvable: When In-Context Search Meets Test-Time Scaling
  5. 2026SearchMaster: Grounded and Regulated Self-Play for Search Agents
  6. 2024Navigating the Labyrinth: Evaluating LLMs' Ability to Reason About Search Problems
  7. 2026Breaking the Evaluation Paradox: Evaluating High-Entropy Search with Computationally Irreducible Constraints
  8. 2025Supervising the search process produces reliable and generalizable information-seeking agents