ENTLORE: A Graph-Grounded Benchmark for Latent Organizational Reasoning in Enterprise Question Answering
Not on the map. Why?
Abstract (arXiv)
Enterprise question answering is framed as retrieving internal documents and generating grounded answers. Routine enterprise records, however, are work by-products in which required organizational relations remain implicit across heterogeneous sources. Existing benchmarks provide realistic multi-source evidence, but often materialize a predefined answer path and therefore test the composition of stated facts rather than recovery of a target relation absent from the corpus. We call the latter capability latent organizational reasoning. We introduce ENTLORE, a graph-grounded benchmark construction framework that reconstructs an audited enterprise world from routine documents, authoritative organizational tables, and operational records. Versioned organizational conventions certify derived relations in a truth graph, enabling complete golden answers and proof certificates. The aligned anonymized release exposes only the document corpus while withholding private structure and target relations. ENTLORE contains 2,341 documents from three source types and 907 questions spanning explicit lookup, cross-source composition, and latent organizational reasoning, evaluated across 56 model and access configurations. Structuring the released world as an induced entity graph or navigable knowledge base gives the strongest deployable results. Yet supplying gold documents still leaves 30.4% of latent questions unanswered, versus 12.6% and 6.2% for explicit and compositional questions. Enterprise QA therefore depends not only on document recall, but also on whether implicit organizational relations become usable. The benchmark, data, and code are publicly available at https://github.com/scitix/entlore .
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 11 Aug 2026.
Builds on
24 of 31 references.
References · 24
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
- 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
- 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
12 more
- 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
- 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
- 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 120
- 2020doc2dial: A Goal-Oriented Document-Grounded Dialogue Datasetcited by 88
- 2020KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Basecited by 76
- 2021MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documentscited by 53
- 2020IIRC: A Dataset of Incomplete Information Reading Comprehension Questionscited by 52
- 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
- 2026EnterpriseRAG-Bench: A RAG Benchmark for Company Internal Knowledgecited by 7
- 2026Navigating Large-Scale Document Collections: MuDABench for Multi-Document Analytical QAcited by 2
- 2026QO-Bench: Diagnosing Query-Operator-Preserving Retrieval over Typed Event Tuplescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.