# Is Grep All You Need? How Agent Harnesses Reshape Agentic Search

arXiv: https://arxiv.org/abs/2605.15184 · HTML: https://mlsanity.com/paper/2605.15184/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 14 May 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 18 of 18)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Sparse, Dense, and Attentional Representations for Text Retrieval](https://mlsanity.com/paper/2005.00181/) (2020)
- [MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations](https://mlsanity.com/paper/2507.21428/) (2025)
- [Chronos: Temporal-Aware Conversational Agents with Structured Event Retrieval for Long-Term Memory](https://mlsanity.com/paper/2603.16862/) (2026)
- [From Rows to Reasoning: A Retrieval-Augmented Multimodal Framework for Spreadsheet Understanding](https://mlsanity.com/paper/2601.08741/) (2026)

## Built on by (most cited first, 21 of 21)

- [GrepSeek: Training Search Agents for Direct Corpus Interaction](https://mlsanity.com/paper/2605.29307/) (2026)
- [Towards Retrieving Interaction Spaces for Agentic Search](https://mlsanity.com/paper/2606.06880/) (2026)
- [Recursive Agent Harnesses](https://mlsanity.com/paper/2606.13643/) (2026)
- [Dr-DCI: Scaling Direct Corpus Interaction via Dynamic Workspace Expansion](https://mlsanity.com/paper/2606.14885/) (2026)
- [Code Isn't Memory: A Structural Codebase Index Inside a Coding Agent](https://mlsanity.com/paper/2606.22417/) (2026)
- [Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study](https://mlsanity.com/paper/2608.01507/) (2026)
- [Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents](https://mlsanity.com/paper/2608.02097/) (2026)
- [WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing](https://mlsanity.com/paper/2609.05171/) (2026)
- [Semantic Triplet Restoration: A Novel Protocol for Hierarchical Table Understanding in Large Language Models](https://mlsanity.com/paper/2605.31550/) (2026)
- [DAR: Deontic Reasoning with Agentic Harnesses](https://mlsanity.com/paper/2606.05009/) (2026)
- [Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution](https://mlsanity.com/paper/2607.08700/) (2026)
- [ETAS: An Effect-Typed Language for Agent Systems](https://mlsanity.com/paper/2607.17780/) (2026)
- [Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations](https://mlsanity.com/paper/2608.06305/) (2026)
- [Projecting BrowseComp-Plus onto ClimbMix: Toward More Realistic Corpora for Agentic Search](https://mlsanity.com/paper/2608.20317/) (2026)
- [Fidelity Is Not Enough: Dispatch-Level Instrumentation for Agentic Datasheet Extraction](https://mlsanity.com/paper/2608.28439/) (2026)
- [PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents](https://mlsanity.com/paper/2609.06702/) (2026)
- [ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research](https://mlsanity.com/paper/2610.02202/) (2026)
- [Harness-Search: Guiding Long-Horizon Search through Multi-Agent Coordination](https://mlsanity.com/paper/2610.05382/) (2026)
- [RunningTab: Direct Workspace Interaction with Environment-Side Tabs](https://mlsanity.com/paper/2610.10444/) (2026)
- [Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfitting](https://mlsanity.com/paper/2610.12601/) (2026)
- [Knowledge Graphs as Files: An Agent Harness Where Tool Creation and Tool Use Co-Evolve](https://mlsanity.com/paper/2610.18616/) (2026)

## Abstract (arXiv)

Recent advances in Large Language Model (LLM) agents have enabled complex agentic workflows where models autonomously retrieve information, call tools, and reason over large corpora to complete tasks on behalf of users. Despite the growing adoption of retrieval-augmented generation (RAG) in agentic search systems, existing literature lacks a systematic comparison of how retrieval strategy choice interacts with agent architecture and tool-calling paradigm. Important practical dimensions, including how tool outputs are presented to the model and how performance changes when searches must cope with more irrelevant surrounding text, remain under-explored in agent loops. This paper reports an empirical study organized into two experiments. Experiment 1 compares grep and vector retrieval on a 116-question sample from LongMemEval, using a custom agent harness (Chronos) and provider-native CLI harnesses (Claude Code, Codex, and Gemini CLI), for both inline tool results and file-based tool results that the model reads separately. Experiment 2 compares grep-only and vector-only retrieval while progressively mixing in additional unrelated conversation history, so that each query is embedded in more distracting material alongside the passages that matter. Across Chronos and the provider CLIs, grep generally yields higher accuracy than vector retrieval in our comparisons in experiment 1; at the same time, overall scores still depend strongly on which harness and tool-calling style is used, even when the underlying conversation data are the same.