# Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Explanation and Verification Agents

arXiv: https://arxiv.org/abs/2507.18607 · HTML: https://mlsanity.com/paper/2507.18607/

## Key facts

- Cited by 4 papers in the map (data through 7 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 36 of its 67 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [Evaluating the Robustness of Neural Language Models to Input Perturbations](https://mlsanity.com/paper/2108.12237/) (2021)
- [Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method](https://mlsanity.com/paper/2305.13412/) (2023)
- [Evaluations and Methods for Explanation through Robustness Analysis](https://mlsanity.com/paper/2006.00442/) (2020)
- [Embedding Comparator: Visualizing Differences in Global Structure and Local Neighborhoods via Small Multiples](https://mlsanity.com/paper/1912.04853/) (2019)
- [Text Summarization Using Large Language Models: A Comparative Study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT Models](https://mlsanity.com/paper/2310.10449/) (2023)
- [VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation](https://mlsanity.com/paper/2406.17681/) (2024)
- [Prompt Perturbation Consistency Learning for Robust Language Models](https://mlsanity.com/paper/2402.15833/) (2024)
- [Statistical Analysis and Parameter Selection for Mapper](https://mlsanity.com/paper/1706.00204/) (2017)
- [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023)
- [Emblaze: Illuminating Machine Learning Representations through Interactive Comparison of Embedding Spaces](https://mlsanity.com/paper/2202.02641/) (2022)
- [embComp: Visual Interactive Comparison of Vector Embeddings](https://mlsanity.com/paper/1911.01542/) (2019)
- [Visual Comparison of Language Model Adaptation](https://mlsanity.com/paper/2208.08176/) (2022)
- [SyntaxShap: Syntax-aware Explainability Method for Text Generation](https://mlsanity.com/paper/2402.09259/) (2024)
- [Local Structure Matters Most: Perturbation Study in NLU](https://mlsanity.com/paper/2107.13955/) (2021)
- [Experimental Observations of the Topology of Convolutional Neural Network Activations](https://mlsanity.com/paper/2212.00222/) (2022)

## Built on by (most cited first, 4 of 4)

- [Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts](https://mlsanity.com/paper/2511.06048/) (2025)
- [Teaching People LLM's Errors and Getting it Right](https://mlsanity.com/paper/2512.21422/) (2025)
- [Visualising Information Flow in Word Embeddings with Diffusion Tensor Imaging](https://mlsanity.com/paper/2601.05713/) (2026)
- [TopoAlign: Topology-Aware Visual Representation Alignment](https://mlsanity.com/paper/2605.25541/) (2026)

## Abstract (arXiv)

Large language models (LLMs) produce high-dimensional embeddings that capture rich semantic and syntactic relationships between words, sentences, and concepts. Investigating the topological structures of LLM embedding spaces via mapper graphs enables us to understand their underlying structures. Specifically, a mapper graph summarizes the topological structure of the embedding space, where each node represents a topological neighborhood (containing a cluster of embeddings), and an edge connects two nodes if their corresponding neighborhoods overlap. However, manually exploring these embedding spaces to uncover encoded linguistic properties requires considerable human effort. To address this challenge, we introduce a framework for semi-automatic annotation of these embedding properties. To organize the exploration process, we first define a taxonomy of explorable elements within a mapper graph such as nodes, edges, paths, components, and trajectories. The annotation of these elements is executed through two types of customizable LLM-based agents that employ perturbation techniques for scalable and automated analysis. These agents help to explore and explain the characteristics of mapper elements and verify the robustness of the generated explanations. We instantiate the framework within a visual analytics workspace and demonstrate its effectiveness through case studies. In particular, we replicate findings from prior research on BERT's embedding properties across various layers of its architecture and provide further observations into the linguistic properties of topological neighborhoods.