# Mapping 1,000+ Language Models via the Log-Likelihood Vector

arXiv: https://arxiv.org/abs/2502.16173 · HTML: https://mlsanity.com/paper/2502.16173/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2025.

## Built on by (most cited first, 7 of 7)

- [Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality](https://mlsanity.com/paper/2506.14681/) (2025)
- [Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings](https://mlsanity.com/paper/2505.15353/) (2025)
- [Likelihood Variance as Text Importance for Resampling Texts to Map Language Models](https://mlsanity.com/paper/2505.15428/) (2025)
- [Map of Encoders -- Mapping Sentence Encoders using Quantum Relative Entropy](https://mlsanity.com/paper/2602.08740/) (2026)
- [Domain Mixture Design via Log-Likelihood Differences for Aligning Language Models with a Target Model](https://mlsanity.com/paper/2603.16622/) (2026)
- [Language Model Maps for Prompt-Response Distributions via Log-Likelihood Vectors](https://mlsanity.com/paper/2603.18593/) (2026)
- [ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding](https://mlsanity.com/paper/2606.07524/) (2026)

## Abstract (arXiv)

To compare autoregressive language models at scale, we propose using log-likelihood vectors computed on a predefined text set as model features. This approach has a solid theoretical basis: when treated as model coordinates, their squared Euclidean distance approximates the Kullback-Leibler divergence of text-generation probabilities. Our method is highly scalable, with computational cost growing linearly in both the number of models and text samples, and is easy to implement as the required features are derived from cross-entropy loss. Applying this method to over 1,000 language models, we constructed a "model map," providing a new perspective on large-scale model analysis.