mlsanityData to 14 Oct 2026

BRIGHT+: Upgrading the BRIGHT Benchmark with MARCUS, a Multi-Agent RAG Clean-Up Suite

arXiv 2506.07116 ↗First posted 8 Jun 2025cs.AIcited by 2

On the map: Retrieval-augmented generation → Agentic RAG, agents and memory

also in 1 more
Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems require corpora that are both structurally clean and semantically coherent. BRIGHT is a recent and influential benchmark designed to evaluate complex multi-hop retrieval across diverse, high-reasoning domains. However, its practical effectiveness is limited by common web-crawled artifacts - such as content redundancy and semantic discontinuity - that impair retrieval accuracy and downstream reasoning. Notably, we find that such issues are concentrated in seven StackExchange-derived subdomains, while other domains (e.g., Coding and Theorem-based content) remain relatively clean. In this study, we present MARCUS, a multi-agent pipeline that leverages large language models (LLMs) to systematically clean and re-chunk BRIGHT into a higher-quality corpus: BRIGHT-Plus. MARCUS applies dedicated agents for structural noise removal and semantic segmentation, preserving answer-bearing spans while improving contextual integrity. Experimental evaluations demonstrate that BRIGHT-Plus yields consistent and significant improvements in both retrieval accuracy and multi-hop reasoning across a diverse set of retrievers. We release both the BRIGHT-Plus corpus and the MARCUS pipeline to support future research on robust, reasoning-centric retrieval.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 9 of its 16 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

9 of 16 references.

References · 9

  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  2. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  3. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  4. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 425
  5. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 294
  6. 2024Generative Representational Instruction Tuningcited by 233
  7. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 158
  8. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  9. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Lighting the Way for BRIGHT: Reproducible Baselines with Anserini, Pyserini, and RankLLM
  2. 2026Agent-Orchestrated Adaptive RAG: A Comparative Study on Structured and Multi-Hop Retrieval
  3. 2026AgenticRAGTracer: A Hop-Aware Benchmark for Diagnosing Multi-Step Retrieval Reasoning in Agentic RAG
  4. 2025HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation
  5. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  6. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces
  7. 2026MiRAGE: A Multiagent Framework for Generating Multimodal Multihop Question-Answer Dataset for RAG Evaluation
  8. 2025MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoning