# EnterpriseRAG-Bench: A RAG Benchmark for Company Internal Knowledge

arXiv: https://arxiv.org/abs/2605.05253 · HTML: https://mlsanity.com/paper/2605.05253/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 5 May 2026.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 10 of 10)

- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool use](https://mlsanity.com/paper/2602.23368/) (2025)

## Built on by (most cited first, 9 of 9)

- [BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms](https://mlsanity.com/paper/2607.26497/) (2026)
- [ENTLORE: A Graph-Grounded Benchmark for Latent Organizational Reasoning in Enterprise Question Answering](https://mlsanity.com/paper/2608.10679/) (2026)
- [LayerRAG-Bench: A Cross-Layer Reliability Benchmark for Agentic Retrieval-Augmented Generation](https://mlsanity.com/paper/2607.27353/) (2026)
- [MegaMem: A Retrieval Solution for Ultra-Large Context Windows](https://mlsanity.com/paper/2608.22137/) (2026)
- [MEMONDEMAND: A Memory Management System for Large-Scale Enterprise Data](https://mlsanity.com/paper/2608.22141/) (2026)
- [Evidence Blindness in Direct Corpus Interaction: Persistent Navigation with AtlasNav](https://mlsanity.com/paper/2608.24764/) (2026)
- [Better Together: Complementary Query Rewriting Under a Strong RAG Baseline](https://mlsanity.com/paper/2609.05637/) (2026)
- [Follow the Entities: A Corpus Map for Agentic Search](https://mlsanity.com/paper/2609.37226/) (2026)
- [CoverSynth: Coverage-Guided Generation of Retrieval Evaluation Sets](https://mlsanity.com/paper/2610.15409/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has become the standard approach for grounding large language models in information that was not available during training. While existing datasets and benchmarks focus on web or other public sources, there is still no widely adopted dataset that realistically reflects the nature of company-internal knowledge. Meanwhile, startups, enterprises, and researchers are increasingly developing AI Agents designed to operate over exactly this kind of proprietary data. To close this gap, we release a synthetic enterprise corpus, its generation framework, and a leaderboard. We present EnterpriseRAG-Bench, a dataset consisting of approximately 500,000 documents spanning nine enterprise source types (Slack, Gmail, Linear, Google Drive, HubSpot, Fireflies, GitHub, Jira, and Confluence) and 500 questions across ten categories that test distinct retrieval and reasoning capabilities. The corpus is generated with cross-document coherence (grounded in shared projects, people, and initiatives) and augmented with realistic noise such as misfiled documents, near-duplicates, and conflicting information. The question set ranges from simple single-document lookups to multi-document reasoning, constrained retrieval, conflict resolution, and recognizing when information is absent. The generation framework lets teams generate variants tailored to their own industry, scale, and source mix. The dataset, code, evaluation harness, and leaderboard are available at https://github.com/onyx-dot-app/EnterpriseRAG-Bench.