# LLM DNA: Tracing Model Evolution via Functional Representations

arXiv: https://arxiv.org/abs/2509.24496 · HTML: https://mlsanity.com/paper/2509.24496/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 42 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Builds on (most cited first, 30 of 32)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack](https://mlsanity.com/paper/2404.01833/) (2024)
- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data](https://mlsanity.com/paper/2410.01560/) (2024)
- [Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents](https://mlsanity.com/paper/2402.11208/) (2024)
- [LLM Multi-Agent Systems: Challenges and Open Problems](https://mlsanity.com/paper/2402.03578/) (2024)
- [A Semantic Invariant Robust Watermark for Large Language Models](https://mlsanity.com/paper/2310.06356/) (2023)
- [Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration](https://mlsanity.com/paper/2404.12715/) (2024)
- [EmbedLLM: Learning Compact Representations of Large Language Models](https://mlsanity.com/paper/2410.02223/) (2024)
- [MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs](https://mlsanity.com/paper/2407.10834/) (2024)
- [LLM-Ensemble: Optimal Large Language Model Ensemble Method for E-commerce Product Attribute Value Extraction](https://mlsanity.com/paper/2403.00863/) (2024)
- [Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends](https://mlsanity.com/paper/2508.11548/) (2025)
- [PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks](https://mlsanity.com/paper/2404.04671/) (2024)
- [Model Provenance Testing for Large Language Models](https://mlsanity.com/paper/2502.00706/) (2025)
- [The Johnson-Lindenstrauss lemma is optimal for linear dimensionality reduction](https://mlsanity.com/paper/1411.2404/) (2014)
- [Transferring Backdoors between Large Language Models by Knowledge Distillation](https://mlsanity.com/paper/2408.09878/) (2024)
- [Independence Tests for Language Models](https://mlsanity.com/paper/2502.12292/) (2025)
- [Quantification of Large Language Model Distillation](https://mlsanity.com/paper/2501.12619/) (2025)
- [RAFP: Identifying LLM Lineages via Rare-Region Fingerprints](https://mlsanity.com/paper/2505.12682/) (2025)
- [HuggingGraph: Understanding the Supply Chain of LLM Ecosystem](https://mlsanity.com/paper/2507.14240/) (2025)
- [Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts](https://mlsanity.com/paper/2508.06361/) (2025)
- [NSmark: Null Space Based Black-box Watermarking Defense Framework for Language Models](https://mlsanity.com/paper/2410.13907/) (2024)
- [Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond](https://mlsanity.com/paper/2402.14522/) (2024)

## Built on by (most cited first, 15 of 15)

- [AttnDiff: Attention-based Differential Fingerprinting for Large Language Models](https://mlsanity.com/paper/2604.05502/) (2026)
- [Your "Pro" LLM Subscription May Actually Be "Free": Exposing Fingerprint Spoofing Risks in LLM Inference Services](https://mlsanity.com/paper/2606.16100/) (2026)
- [Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guarantees](https://mlsanity.com/paper/2606.19376/) (2026)
- [LOCUS: Low-Dimensional Model Embeddings for Efficient Model Exploration, Comparison, and Selection](https://mlsanity.com/paper/2601.21082/) (2026)
- [Provable Model Provenance Set for Large Language Models](https://mlsanity.com/paper/2602.00772/) (2026)
- [ModelLens: Finding the Best for Your Task from Myriads of Models](https://mlsanity.com/paper/2605.07075/) (2026)
- [Which Models Are Our Models Built On? Auditing Invisible Dependencies in Modern LLMs](https://mlsanity.com/paper/2606.12385/) (2026)
- [ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding](https://mlsanity.com/paper/2606.07524/) (2026)
- [AI Supply Chain Galaxy: 3D Visual Analytics for License Compliance](https://mlsanity.com/paper/2606.16292/) (2026)
- [Spectral Signatures of Large Language Models](https://mlsanity.com/paper/2607.03377/) (2026)
- [Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges](https://mlsanity.com/paper/2607.28636/) (2026)
- [Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space](https://mlsanity.com/paper/2608.07786/) (2026)
- [Mapping and Measuring the Behavioral Evolution of Large Language Models](https://mlsanity.com/paper/2608.11027/) (2026)
- [Target-Independent Micro-Interventions for Predicting Training Response Across Language-Model Families](https://mlsanity.com/paper/2609.08618/) (2026)
- [Witness Overlap: Directional Provenance Inside Open-Weight Model Families](https://mlsanity.com/paper/2609.31784/) (2026)

## Abstract (arXiv)

The explosive growth of large language models (LLMs) has created a vast but opaque landscape: millions of models exist, yet their evolutionary relationships through fine-tuning, distillation, or adaptation are often undocumented or unclear, complicating LLM management. Existing methods are limited by task specificity, fixed model sets, or strict assumptions about tokenizers or architectures. Inspired by biological DNA, we address these limitations by mathematically defining LLM DNA as a low-dimensional, bi-Lipschitz representation of functional behavior. We prove that LLM DNA satisfies inheritance and genetic determinism properties and establish the existence of DNA. Building on this theory, we derive a general, scalable, training-free pipeline for DNA extraction. In experiments across 305 LLMs, DNA aligns with prior studies on limited subsets and achieves superior or competitive performance on specific tasks. Beyond these tasks, DNA comparisons uncover previously undocumented relationships among LLMs. We further construct the evolutionary tree of LLMs using phylogenetic algorithms, which align with shifts from encoder-decoder to decoder-only architectures, reflect temporal progression, and reveal distinct evolutionary speeds across LLM families.