Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification
Not on the map. Why?
Abstract (arXiv)
Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented. We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry? Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry. We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints. On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 14 Aug 2026.
Builds on
29 of 44 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2016Gaussian Error Linear Units (GELUs)cited by 3,502
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2020GLU Variants Improve Transformercited by 1,804
17 more
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
- 2018Model Cards for Model Reportingcited by 1,080
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
- 2024Yi: Open Foundation Models by 01.AIcited by 719
- 2023Baichuan 2: Open Large-scale Language Modelscited by 670
- 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
- 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
- 2024InternLM2 Technical Reportcited by 397
- 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 357
- 2017Embedding Watermarks into Deep Neural Networkscited by 248
- 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
- 2017Adversarial Frontier Stitching for Remote Neural Network Watermarkingcited by 118
- 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 104
- 2021Proof-of-Learning: Definitions and Practicecited by 84
- 2019IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundarycited by 70
- 2021SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)cited by 69
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning
- 2026TokenPrint: A Calibrated Token-Space Fingerprint for Language-Model Provenance
- 2025RAFP: Identifying LLM Lineages via Rare-Region Fingerprints
- 2025Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures
- 2026The Truth Stays in the Family: Enhancing Contextual Grounding via Inherited Truthful Heads in Model Lineages
- 2026Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs
- 2025TRACEALIGN -- Tracing the Drift: Attributing Alignment Failures to Training-Time Belief Sources in LLMs
- 2026Fail-Closed Alignment for Large Language Models