# Reversing the Lens: Using Explainable AI to Understand Human Expertise

arXiv: https://arxiv.org/abs/2510.13814 · HTML: https://mlsanity.com/paper/2510.13814/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 42 references that are in the map.
- First posted on arXiv 6 Sep 2025.

## Builds on (most cited first, 6 of 6)

- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Community detection in networks: A user guide](https://mlsanity.com/paper/1608.00163/) (2016)
- [On Benchmarking Human-Like Intelligence in Machines](https://mlsanity.com/paper/2502.20502/) (2025)
- [Network Models of Expertise in the Complex Task of Operating Particle Accelerators](https://mlsanity.com/paper/2412.17988/) (2024)

## Abstract (arXiv)

Both humans and machine learning models learn from experience, particularly in safety- and reliability-critical domains. While psychology seeks to understand human cognition, the field of Explainable AI (XAI) develops methods to interpret machine learning models. This study bridges these domains by applying computational tools from XAI to analyze human learning. We modeled human behavior during a complex real-world task -- tuning a particle accelerator -- by constructing graphs of operator subtasks. Applying techniques such as community detection and hierarchical clustering to archival operator data, we reveal how operators decompose the problem into simpler components and how these problem-solving structures evolve with expertise. Our findings illuminate how humans develop efficient strategies in the absence of globally optimal solutions, and demonstrate the utility of XAI-based methods for quantitatively studying human cognition.