# Browsing Lost Unformed Recollections: A Benchmark for Tip-of-the-Tongue Search and Reasoning

arXiv: https://arxiv.org/abs/2503.19193 · HTML: https://mlsanity.com/paper/2503.19193/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks](https://mlsanity.com/paper/2411.04468/) (2024)
- [What Will it Take to Fix Benchmarking in Natural Language Understanding?](https://mlsanity.com/paper/2104.02145/) (2021)
- [OpenAGI: When LLM Meets Domain Experts](https://mlsanity.com/paper/2304.04370/) (2023)
- [AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?](https://mlsanity.com/paper/2407.15711/) (2024)
- [InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks](https://mlsanity.com/paper/2401.05507/) (2024)
- [ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness](https://mlsanity.com/paper/2304.10703/) (2023)
- [Rethinking Model Evaluation as Narrowing the Socio-Technical Gap](https://mlsanity.com/paper/2306.03100/) (2023)
- [Towards Ecologically Valid Research on Language User Interfaces](https://mlsanity.com/paper/2007.14435/) (2020)
- [ECBD: Evidence-Centered Benchmark Design for NLP](https://mlsanity.com/paper/2406.08723/) (2024)
- [Decomposing Complex Queries for Tip-of-the-tongue Retrieval](https://mlsanity.com/paper/2305.15053/) (2023)
- [CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrieval](https://mlsanity.com/paper/2405.15784/) (2024)
- [Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification](https://mlsanity.com/paper/2101.07124/) (2021)
- [Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair](https://mlsanity.com/paper/2111.08181/) (2021)
- [Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits](https://mlsanity.com/paper/2403.14791/) (2024)
- [When the Music Stops: Tip-of-the-Tongue Retrieval for Music](https://mlsanity.com/paper/2305.14072/) (2023)

## Built on by (most cited first, 7 of 7)

- [TRAIL: Trace Reasoning and Agentic Issue Localization](https://mlsanity.com/paper/2505.08638/) (2025)
- [A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges](https://mlsanity.com/paper/2508.05668/) (2025)
- [A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications](https://mlsanity.com/paper/2510.16724/) (2025)
- [InteractComp: Evaluating Search Agents With Ambiguous Queries](https://mlsanity.com/paper/2510.24668/) (2025)
- [UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking](https://mlsanity.com/paper/2603.08117/) (2026)
- [DETOUR: An Interactive Benchmark for Dual-Agent Search and Reasoning](https://mlsanity.com/paper/2602.00352/) (2026)
- [Multilingual and Domain-Agnostic Tip-of-the-Tongue Query Generation for Simulated Evaluation](https://mlsanity.com/paper/2604.21096/) (2026)

## Abstract (arXiv)

We introduce Browsing Lost Unformed Recollections, a tip-of-the-tongue known-item search and reasoning benchmark for general AI assistants. BLUR introduces a set of 573 real-world validated questions that demand searching and reasoning across multi-modal and multilingual inputs, as well as proficient tool use, in order to excel on. Humans easily ace these questions (scoring on average 98%), while the best-performing system scores around 56%. To facilitate progress toward addressing this challenging and aspirational use case for general AI assistants, we release 350 questions through a public leaderboard, retain the answers to 250 of them, and have the rest as a private test set.