# OneEval: Benchmarking LLM Knowledge-intensive Reasoning over Diverse Knowledge Bases

arXiv: https://arxiv.org/abs/2506.12577 · HTML: https://mlsanity.com/paper/2506.12577/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 1 of its 38 references that are in the map.
- First posted on arXiv 14 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 1 of 1)

- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)

## Built on by (most cited first, 4 of 4)

- [Environment-Aware Code Generation: How far are We?](https://mlsanity.com/paper/2601.12262/) (2026)
- [KCoEvo: A Knowledge Graph Augmented Framework for Evolutionary Code Generation](https://mlsanity.com/paper/2603.07581/) (2026)
- [StressEval: Failure-Driven Dynamic Benchmarking for Knowledge-Intensive Reasoning in Large Language Models](https://mlsanity.com/paper/2605.01939/) (2026)
- [Towards reliable multimodal disaster severity assessment through preference optimization and explainable vision-language reasoning](https://mlsanity.com/paper/2609.00879/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated substantial progress on reasoning tasks involving unstructured text, yet their capabilities significantly deteriorate when reasoning requires integrating structured external knowledge such as knowledge graphs, code snippets, or formal logic. This limitation is partly due to the absence of benchmarks capable of systematically evaluating LLM performance across diverse structured knowledge modalities. To address this gap, we introduce textbftextscOneEval, a comprehensive benchmark explicitly designed to assess the knowledge-intensive reasoning capabilities of LLMs across four structured knowledge modalities, unstructured text, knowledge graphs, code, and formal logic, and five critical domains (general knowledge, government, science, law, and programming). textscOneEval comprises 4,019 carefully curated instances and includes a challenging subset, textscOneEvaltextsubscriptHard, consisting of 1,285 particularly difficult cases. Through extensive evaluation of 18 state-of-the-art open-source and proprietary LLMs, we establish three core findings: a) persistent limitations in structured reasoning, with even the strongest model achieving only 32.2% accuracy on textscOneEvaltextsubscriptHard; b) performance consistently declines as the structural complexity of the knowledge base increases, with accuracy dropping sharply from 53% (textual reasoning) to 25% (formal logic); and c) diminishing returns from extended reasoning chains, highlighting the critical need for models to adapt reasoning depth appropriately to task complexity. We release the textscOneEval datasets, evaluation scripts, and baseline results publicly, accompanied by a leaderboard to facilitate ongoing advancements in structured knowledge reasoning.