mlsanityData to 14 Oct 2026

SearchJev: A Fast and Calibrated System-1 Model for Search Agents

arXiv 2610.05107 ↗First posted 4 Oct 2026cs.IR cs.CLcited by 1

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Search agents repeatedly make short decisions about relevance, evidence sufficiency, and search actions. Using generative language models for these decisions introduces latency and unreliable confidence. We present SearchJev, a fast and calibrated System-1 model that separates search decisions from System-2 reasoning and generation. Given a search state and a decision schema, SearchJev directly scores legal options without autoregressive output generation. We propose Soft-Label Learning for Calibrated Decisions (SLCD) to learn decision probabilities from uncertain supervision and calibrate their confidence. In a dual-system search agent, SearchJev handles short decisions and delegates uncertain judgments to System 2, which retains planning, query generation, and answer composition. We also introduce SearchDecision-Bench, a benchmark unifying six types of search decisions for training and evaluation. On SearchDecision-Bench, SEARCHJEV improves decision quality over same-size Qwen3.5 autoregressive models, achieves 5.2-5.3 times faster decisions, and reduces average expected calibration error by 41-74%. On BrowseComp-Plus, the dual-system agents achieve a 3.7-4.7 times speedup in active search time while improving answer accuracy from 45% to up to 54%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 59 of its 90 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

59 of 90 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery9 Mar
  2. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories11 Apr
  3. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 56

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  6. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  9. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  12. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
38 more
  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  2. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  3. 2017Selective Classification for Deep Neural Networkscited by 852
  4. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  5. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  6. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  7. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  8. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  9. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 402
  10. 2022Using cognitive psychology to understand GPT-3cited by 398
  11. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  12. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 376
  13. 2020Overview of the TREC 2019 deep learning trackcited by 286
  14. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  15. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
  16. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
  17. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 236
  18. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 225
  19. 2021Overview of the TREC 2020 deep learning trackcited by 196
  20. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 162
  21. 2023RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!cited by 155
  22. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 148
  23. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  24. 2020The Multilingual Amazon Reviews Corpuscited by 143
  25. 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 101
  26. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 85
  27. 2025HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languagescited by 75
  28. 2023T2Ranking: A large-scale Chinese Benchmark for Passage Rankingcited by 60
  29. 2019Improving Multi-turn Dialogue Modelling with Utterance ReWritercited by 58
  30. 2025DeepShop: A Benchmark for Deep Research Shopping Agentscited by 46
  31. 2025When2Call: When (not) to Call Toolscited by 42
  32. 2025Search Arena: Analyzing Search-Augmented LLMscited by 34
  33. 2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 27
  34. 2025Deep Research: A Systematic Surveycited by 27
  35. 2024MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimizationcited by 22
  36. 2023Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoningcited by 19
  37. 2025GLiClass: Generalist Lightweight Model for Sequence Classification Taskscited by 15
  38. 2017On Measuring Bias in Online Informationcited by 8

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SwarmRSI: Recursive Self-Improvement through Evolving Scientific Agent Swarmcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task
  2. 2025Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty
  3. 2025ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards
  4. 2026CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search
  5. 2025Supervising the search process produces reliable and generalizable information-seeking agents
  6. 2026MemRetriever: Learning to Search, Reflect, and Retrieve from Long-Term Memory ↗
  7. 2026SE-Search: Self-Evolving Search Agent via Memory and Dense Reward
  8. 2026IGSD: Environment-Verified Hindsight Self-Distillation for Search Agents