mlsanityData to 14 Oct 2026

DelistBench: Evaluating Search-Enabled LLMs for Auditable Corporate-Event Database Completion

arXiv 2608.22770 ↗First posted 24 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Financial institutions need an independent way to detect missing, stale, and misclassified corporate-event records in vendor databases. We introduce Search-to-Record, a database-assurance task in which search-enabled large language models reconstruct institution-defined event records from public sources for a known security universe and historical cutoff, and DelistBench, a 1,200-record benchmark for security-level delisting announcements. We evaluate five models in paired closed-book and web-enabled conditions. Web access raises announcement-date accuracy within seven days by 34.0 to 48.0 percentage points and event-status accuracy by approximately 2.8 to 21.7 points; the best system achieves 81.5% overall joint accuracy within seven days. Economy web systems achieve 75.9-78.3% overall joint accuracy within seven days at 4.5-6.6% of the API cost of the most expensive web system. Risk-based triage identifies low-error subsets, although the highest-coverage operating point still sends 27.3% of the balanced test set to review. The evaluation identifies web retrieval as the main source of timing gains and shows that low-cost systems can approach the best system's accuracy. Together, Search-to-Record, DelistBench, and the evaluation provide concrete deployment guidance: calibrate triage to local event prevalence and market mix, preserve positive-event recall, and route positive and ambiguous cases to targeted review.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 14 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

8 of 14 references.

References · 8

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2017Selective Classification for Deep Neural Networkscited by 852
  3. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  4. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  5. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
  6. 2022RealTime QA: What's the Answer Right Now?cited by 244
  7. 2025FinDeepResearch: Evaluating Deep Research Agents in Rigorous Financial Analysiscited by 14
  8. 2026FinDeepForecast: A Live Multi-Agent System for Benchmarking Deep Research Agents in Financial Forecastingcited by 4

Built on by

Not cited yet.