# Intrinsic Fingerprint of LLMs: Continue Training is NOT All You Need to Steal A Model!

arXiv: https://arxiv.org/abs/2507.03014 · HTML: https://mlsanity.com/paper/2507.03014/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Builds on (most cited first, 8 of 8)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [OLMoE: Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2409.02060/) (2024)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [Llama-Nemotron: Efficient Reasoning Models](https://mlsanity.com/paper/2505.00949/) (2025)
- [Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity](https://mlsanity.com/paper/2505.21411/) (2025)
- [Can Distillation Mitigate Backdoor Attacks in Pre-trained Encoders?](https://mlsanity.com/paper/2403.03846/) (2024)

## Built on by (most cited first, 18 of 18)

- [Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends](https://mlsanity.com/paper/2508.11548/) (2025)
- [SoK: Large Language Model Copyright Auditing via Fingerprinting](https://mlsanity.com/paper/2508.19843/) (2025)
- [SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained From](https://mlsanity.com/paper/2509.26404/) (2025)
- [Fingerprinting LLMs via Prompt Injection](https://mlsanity.com/paper/2509.25448/) (2025)
- [Are Robust LLM Fingerprints Adversarially Robust?](https://mlsanity.com/paper/2509.26598/) (2025)
- [On Protecting Agentic Systems' Intellectual Property via Watermarking](https://mlsanity.com/paper/2602.08401/) (2026)
- [AttnDiff: Attention-based Differential Fingerprinting for Large Language Models](https://mlsanity.com/paper/2604.05502/) (2026)
- [PathMark: Protecting Intellectual Property of Mixture-of-Expert LLMs via Path Watermarks](https://mlsanity.com/paper/2607.03688/) (2026)
- [AWM: Accurate Weight-Matrix Fingerprint for Large Language Models](https://mlsanity.com/paper/2510.06738/) (2025)
- [FLIPS: Instance-Fingerprinting for LLMs via Pseudo-random Sequences](https://mlsanity.com/paper/2606.03330/) (2026)
- [Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures](https://mlsanity.com/paper/2511.06390/) (2025)
- [Matrix-Driven Identification and Reconstruction of LLM Weight Homology](https://mlsanity.com/paper/2508.06309/) (2025)
- [Provable Model Provenance Set for Large Language Models](https://mlsanity.com/paper/2602.00772/) (2026)
- [Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Content](https://mlsanity.com/paper/2605.29245/) (2026)
- [Spectral Signatures of Large Language Models](https://mlsanity.com/paper/2607.03377/) (2026)
- [modelDNA: Calibrated Lineage Verification and Merge Decomposition from Sampled Weight Fingerprints](https://mlsanity.com/paper/2607.10617/) (2026)
- [Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space](https://mlsanity.com/paper/2608.07786/) (2026)
- [Fingerprinting Multimodal Large Language Models](https://mlsanity.com/paper/2609.20457/) (2026)

## Abstract (arXiv)

Large language models (LLMs) face significant copyright and intellectual property challenges as the cost of training increases and model reuse becomes prevalent. While watermarking techniques have been proposed to protect model ownership, they may not be robust to continue training and development, posing serious threats to model attribution and copyright protection. This work introduces a simple yet effective approach for robust LLM fingerprinting based on intrinsic model characteristics. We discover that the standard deviation distributions of attention parameter matrices across different layers exhibit distinctive patterns that remain stable even after extensive continued training. These parameter distribution signatures serve as robust fingerprints that can reliably identify model lineage and detect potential copyright infringement. Our experimental validation across multiple model families demonstrates the effectiveness of our method for model authentication. Notably, our investigation uncovers evidence that a recently Pangu Pro MoE model released by Huawei is derived from Qwen-2.5 14B model through upcycling techniques rather than training from scratch, highlighting potential cases of model plagiarism, copyright violation, and information fabrication. These findings underscore the critical importance of developing robust fingerprinting methods for protecting intellectual property in large-scale model development and emphasize that deliberate continued training alone is insufficient to completely obscure model origins.