# PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks

arXiv: https://arxiv.org/abs/2404.04671 · HTML: https://mlsanity.com/paper/2404.04671/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 41 references that are in the map.
- First posted on arXiv 6 Apr 2024.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [Arcee's MergeKit: A Toolkit for Merging Large Language Models](https://mlsanity.com/paper/2403.13257/) (2024)
- [Multi-lingual Evaluation of Code Generation Models](https://mlsanity.com/paper/2210.14868/) (2022)
- [Investigating Data Contamination in Modern Benchmarks for Large Language Models](https://mlsanity.com/paper/2311.09783/) (2023)
- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond](https://mlsanity.com/paper/2309.16583/) (2023)

## Built on by (most cited first, 19 of 19)

- [We Should Chart an Atlas of All the World's Models](https://mlsanity.com/paper/2503.10633/) (2025)
- [Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends](https://mlsanity.com/paper/2508.11548/) (2025)
- [LLM DNA: Tracing Model Evolution via Functional Representations](https://mlsanity.com/paper/2509.24496/) (2025)
- [Independence Tests for Language Models](https://mlsanity.com/paper/2502.12292/) (2025)
- [From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models](https://mlsanity.com/paper/2407.09502/) (2024)
- [Shared Imagination: LLMs Hallucinate Alike](https://mlsanity.com/paper/2407.16604/) (2024)
- [Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research](https://mlsanity.com/paper/2509.22831/) (2025)
- [LogProber: Disentangling confidence from contamination in LLM responses](https://mlsanity.com/paper/2408.14352/) (2024)
- [Likelihood Variance as Text Importance for Resampling Texts to Map Language Models](https://mlsanity.com/paper/2505.15428/) (2025)
- [Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs](https://mlsanity.com/paper/2602.16085/) (2026)
- [No Such Thing as a General Learner: Language models and their dual optimization](https://mlsanity.com/paper/2408.09544/) (2024)
- [Discovering Hidden Gems in Model Repositories](https://mlsanity.com/paper/2601.22157/) (2026)
- [Which Models Are Our Models Built On? Auditing Invisible Dependencies in Modern LLMs](https://mlsanity.com/paper/2606.12385/) (2026)
- [TokenPrint: A Calibrated Token-Space Fingerprint for Language-Model Provenance](https://mlsanity.com/paper/2608.08139/) (2026)
- [Language Model Maps for Prompt-Response Distributions via Log-Likelihood Vectors](https://mlsanity.com/paper/2603.18593/) (2026)
- [Analysis and Explainability of LLMs Via Evolutionary Methods](https://mlsanity.com/paper/2605.02930/) (2026)
- [ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding](https://mlsanity.com/paper/2606.07524/) (2026)
- [Post-Training Recipe, More Than Model Family, Shapes Multi-Agent LLM Conversational Behavior](https://mlsanity.com/paper/2606.20632/) (2026)
- [Retrospective Distillation Attribution via Normalized Response Similarity](https://mlsanity.com/paper/2609.32749/) (2026)

## Abstract (arXiv)

This paper introduces PhyloLM, a method adapting phylogenetic algorithms to Large Language Models (LLMs) to explore whether and how they relate to each other and to predict their performance characteristics. Our method calculates a phylogenetic distance metric based on the similarity of LLMs' output. The resulting metric is then used to construct dendrograms, which satisfactorily capture known relationships across a set of 111 open-source and 45 closed models. Furthermore, our phylogenetic distance predicts performance in standard benchmarks, thus demonstrating its functional validity and paving the way for a time and cost-effective estimation of LLM capabilities. To sum up, by translating population genetic concepts to machine learning, we propose and validate a tool to evaluate LLM development, relationships and capabilities, even in the absence of transparent training information.