# Explainable Benchmarking through the Lense of Concept Learning

arXiv: https://arxiv.org/abs/2510.20439 · HTML: https://mlsanity.com/paper/2510.20439/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 39 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Builds on (most cited first, 6 of 6)

- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [QED: A Framework and Dataset for Explanations in Question Answering](https://mlsanity.com/paper/2009.06354/) (2020)
- [QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata Translated by Native Speakers](https://mlsanity.com/paper/2202.00120/) (2022)
- [EvoLearner: Learning Description Logics with Evolutionary Algorithms](https://mlsanity.com/paper/2111.04879/) (2021)
- [Learning Concept Lengths Accelerates Concept Learning in ALC](https://mlsanity.com/paper/2107.04911/) (2021)
- [Knowledge Graph Question Answering using Graph-Pattern Isomorphism](https://mlsanity.com/paper/2103.06752/) (2021)

## Abstract (arXiv)

Evaluating competing systems in a comparable way, i.e., benchmarking them, is an undeniable pillar of the scientific method. However, system performance is often summarized via a small number of metrics. The analysis of the evaluation details and the derivation of insights for further development or use remains a tedious manual task with often biased results. Thus, this paper argues for a new type of benchmarking, which is dubbed explainable benchmarking. The aim of explainable benchmarking approaches is to automatically generate explanations for the performance of systems in a benchmark. We provide a first instantiation of this paradigm for knowledge-graph-based question answering systems. We compute explanations by using a novel concept learning approach developed for large knowledge graphs called PruneCEL. Our evaluation shows that PruneCEL outperforms state-of-the-art concept learners on the task of explainable benchmarking by up to 0.55 points F1 measure. A task-driven user study with 41 participants shows that in 80% of the cases, the majority of participants can accurately predict the behavior of a system based on our explanations. Our code and data are available at https://github.com/dice-group/PruneCEL/tree/K-cap2025