mlsanityData to 14 Oct 2026

Ko-WideSearch: A Korean Breadth-Search Benchmark for Exhaustive Set Enumeration by Web Agents

arXiv 2606.27595 ↗First posted 25 Jun 2026cs.CLcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Web-agent benchmarks overwhelmingly measure depth -- pinning one obscure answer behind a chain of constraints -- while breadth, exhaustively enumerating a closed set and filling each item's attributes, is barely evaluated, especially outside English. Breadth is also hard to build: certifying that a gold set is complete and every cell correct is far costlier than checking a single answer. I introduce textscKo-WideSearch, a Korean breadth-search benchmark built by an automated synthesize-and-verify pipeline. Each task names a set-parent entity -- a TV season, a dynasty, a league, an administrative region, an election -- and asks for its full membership plus a per-item attribute table, graded by Item-, Column-, and Row-F1. It spans 228 tables over 190 entities and sixteen categories across three difficulty tiers, set by two structural knobs I dial independently -- table width and a 2-D composite key -- so cross-product membership climbs from 0% to 100% across the tiers. A single normalization-aware comparator is shared between gold construction and grading, so stable date and count columns are not over-dropped on formatting alone. Across twenty web agents, the failure is consistent: agents recover the set but not the rows (e.g. Item-F1 92.8 against Row-F1 53.7), accuracy falls steadily as the knobs harden, and neither more search nor more spend closes the gap. Broken down by cell, the hard part is finding the right value, not formatting it: open-ended free-text cells fail most, while cells with a standard answer such as a date or a name usually come out right.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 42 of its 55 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

42 of 55 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts1 Jun
  2. 2025WideSearch: Benchmarking Agentic Broad Info-Seeking11 Aug
  3. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?22 Jul
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
  5. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
  6. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 36

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  9. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  12. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
24 more
  1. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  2. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  3. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  4. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  5. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  6. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
  7. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  8. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  9. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  10. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  11. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  12. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 264
  13. 2023Proving Test Set Contamination in Black Box Language Modelscited by 230
  14. 2021FeTaQA: Free-form Table Question Answeringcited by 175
  15. 2021KLUE: Korean Language Understanding Evaluationcited by 114
  16. 2024KMMLU: Measuring Massive Multitask Language Understanding in Koreancited by 91
  17. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 86
  18. 2021What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformerscited by 76
  19. 2024CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Koreancited by 66
  20. 2023HAE-RAE Bench: Evaluation of Korean Knowledge in Language Modelscited by 47
  21. 2022KOBEST: Korean Balanced Evaluation of Significant Taskscited by 37
  22. 2024EXAONE 3.0 7.8B Instruction Tuned Language Modelcited by 21
  23. 2025Kanana: Compute-efficient Bilingual Language Modelscited by 20
  24. 2026Soft Contamination Means Benchmarks Test Shallow Generalizationcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023EFOk-CQA: Towards Knowledge Graph Complex Query Answering beyond Set Operation
  2. 2025Improving Table Understanding with LLMs and Entity-Oriented Search
  3. 2026BlitzRank: Principled Zero-shot Ranking Agents with Tournament Graphs
  4. 2026SemJoin: Semantic Join Optimization
  5. 2026ChartWalker: Benchmarking the Cross-Chart RAG Task with Hierarchical Knowledge Graphs
  6. 2024BEAVER: An Enterprise Benchmark for Text-to-SQL
  7. 2026Compositional Chain-of-Relations for Faithful Knowledge Graph Question Answering with Large Language Models
  8. 2026Atlas: Efficient Verifiable Semantic Search