# LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services

arXiv: https://arxiv.org/abs/2512.07436 · HTML: https://mlsanity.com/paper/2512.07436/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 32 of its 81 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 32)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://mlsanity.com/paper/2011.03088/) (2020)
- [From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review](https://mlsanity.com/paper/2504.19678/) (2025)
- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools](https://mlsanity.com/paper/2502.04644/) (2025)
- [Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge](https://mlsanity.com/paper/2506.21506/) (2025)
- [SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models](https://mlsanity.com/paper/2506.01062/) (2025)
- [Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents](https://mlsanity.com/paper/2405.16533/) (2024)
- [SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis](https://mlsanity.com/paper/2505.16834/) (2025)
- [EvolveSearch: An Iterative Self-Evolving Search Agent](https://mlsanity.com/paper/2505.22501/) (2025)
- [Pre-trained Language Model for Web-scale Retrieval in Baidu Search](https://mlsanity.com/paper/2106.03373/) (2021)
- [Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought](https://mlsanity.com/paper/2505.15431/) (2025)
- [Promoting Efficient Reasoning with Verifiable Stepwise Reward](https://mlsanity.com/paper/2508.10293/) (2025)
- [DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking](https://mlsanity.com/paper/2510.20168/) (2025)
- [RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use](https://mlsanity.com/paper/2509.06980/) (2025)
- [MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models](https://mlsanity.com/paper/2406.11288/) (2024)
- [Who Stole Your Data? A Method for Detecting Unauthorized RAG Theft](https://mlsanity.com/paper/2510.07728/) (2025)
- [Queries Are Not Alone: Clustering Text Embeddings for Video Search](https://mlsanity.com/paper/2510.07720/) (2025)
- [Conversational Search: From Fundamentals to Frontiers in the LLM Era](https://mlsanity.com/paper/2506.10635/) (2025)
- [MRGRP: Empowering Courier Route Prediction in Food Delivery Service with Multi-Relational Graph](https://mlsanity.com/paper/2505.11999/) (2025)

## Built on by (most cited first, 8 of 8)

- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [ToolForge: A Data Synthesis Pipeline for Multi-Hop Search without Real-World APIs](https://mlsanity.com/paper/2512.16149/) (2025)
- [VistaHop: Benchmarking Long-Horizon Visual DeepSearch](https://mlsanity.com/paper/2606.03273/) (2026)
- [Deep Research as Rubric for Reinforcement Learning](https://mlsanity.com/paper/2606.01091/) (2026)
- [LocalSUG: City-Preference-Enhanced LLM for Query Suggestion in Local-Life Services](https://mlsanity.com/paper/2603.04946/) (2026)
- [RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation](https://mlsanity.com/paper/2610.11183/) (2026)
- [UrbanAgent: A Tool-Augmented Agent for Cross-System Urban Tasks](https://mlsanity.com/paper/2608.03018/) (2026)
- [WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution](https://mlsanity.com/paper/2610.18067/) (2026)

## Abstract (arXiv)

Recent advances in large reasoning models LRMs have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explores vertical domains with unique challenges. In this work, we focus on local life services and introduce LocalSearchBench, which encompass diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, LocalSearchBench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed LocalPlayground, a unified environment integrating multiple tools for LRMs interaction. Experiments show that even state-of-the-art LRMs struggle on LocalSearchBench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services. Code, Benchmark, and Leaderboard are available at https://localsearchbench.github.io/.