# ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding

arXiv: https://arxiv.org/abs/2606.07524 · HTML: https://mlsanity.com/paper/2606.07524/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 70 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [Guided Integrated Gradients: An Adaptive Path Method for Removing Noise](https://mlsanity.com/paper/2106.09788/) (2021)
- [Language Models are General-Purpose Interfaces](https://mlsanity.com/paper/2206.06336/) (2022)
- [Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arena](https://mlsanity.com/paper/2406.07545/) (2024)
- [Incorporating Priors with Feature Attribution on Text Classification](https://mlsanity.com/paper/1906.08286/) (2019)
- [Visualizing Deep Networks by Optimizing with Integrated Gradients](https://mlsanity.com/paper/1905.00954/) (2019)
- [Physical Symmetries Embedded in Neural Networks](https://mlsanity.com/paper/1904.08991/) (2019)
- [Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution](https://mlsanity.com/paper/2405.04825/) (2024)
- [Gradient based Feature Attribution in Explainable AI: A Technical Review](https://mlsanity.com/paper/2403.10415/) (2024)
- [We Should Chart an Atlas of All the World's Models](https://mlsanity.com/paper/2503.10633/) (2025)
- [PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks](https://mlsanity.com/paper/2404.04671/) (2024)
- [LLM DNA: Tracing Model Evolution via Functional Representations](https://mlsanity.com/paper/2509.24496/) (2025)
- [Transferring Backdoors between Large Language Models by Knowledge Distillation](https://mlsanity.com/paper/2408.09878/) (2024)
- [Ecosystem of Large Language Models for Code](https://mlsanity.com/paper/2405.16746/) (2024)

## Abstract (arXiv)

The explosive growth of large language models (LLMs) has created a heterogeneous and poorly documented ecosystem, making systematic model comparison increasingly important for provenance auditing, security analysis, and model selection. Existing representation methods struggle to address this setting efficiently. Approaches analyzing internal parameters are powerful when architectures are compatible, but face scalability barriers under structural heterogeneity, while methods relying on external outputs may conflate models with similar behaviors and are difficult to align in richer output spaces across different tokenizers. To bridge this gap, we propose ABLE (Attribution-Based Large-model Embedding), a framework that leverages the interpretability space to construct model representations. By aggregating gradient-based feature attributions via a tokenizer-agnostic word-level alignment, ABLE captures model-specific input-sensitivity patterns rather than only surface-level outputs. Beyond empirical utility, we provide a stability analysis showing that, under standard regularity assumptions for differentiable Transformer-style models, ABLE induces a Lipschitz-continuous parameter-to-embedding map with finite-sample convergence guarantees. Extensive experiments on 239 open-source LLMs demonstrate that our training-free approach achieves competitive or superior performance in relation prediction, model routing, and benchmark score prediction.