# Verifying LLM Inference to Detect Model Weight Exfiltration

arXiv: https://arxiv.org/abs/2511.02620 · HTML: https://mlsanity.com/paper/2511.02620/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Builds on (most cited first, 17 of 17)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Undetectable Watermarks for Language Models](https://mlsanity.com/paper/2306.09194/) (2023)
- [Publicly-Detectable Watermarking for Language Models](https://mlsanity.com/paper/2310.18491/) (2023)
- [Learning Harmonized Representations for Speculative Sampling](https://mlsanity.com/paper/2408.15766/) (2024)
- [Planting Undetectable Backdoors in Machine Learning Models](https://mlsanity.com/paper/2204.06974/) (2022)
- [Model Equality Testing: Which Model Is This API Serving?](https://mlsanity.com/paper/2410.20247/) (2024)
- [Pseudorandom Error-Correcting Codes](https://mlsanity.com/paper/2402.09370/) (2024)
- [Watermarking Language Models for Many Adaptive Users](https://mlsanity.com/paper/2405.11109/) (2024)
- [Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test](https://mlsanity.com/paper/2506.06975/) (2025)
- [TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference](https://mlsanity.com/paper/2501.16007/) (2025)
- [Edit Distance Robust Watermarks via Indexing Pseudorandom Codes](https://mlsanity.com/paper/2406.02633/) (2024)

## Built on by (most cited first, 6 of 6)

- [Bit-Exact AI Inference Verification Without Performance Tradeoffs](https://mlsanity.com/paper/2606.00279/) (2026)
- [DiFR: Inference Verification Despite Nondeterminism](https://mlsanity.com/paper/2511.20621/) (2025)
- [Aggressive Compression Enables LLM Weight Theft](https://mlsanity.com/paper/2601.01296/) (2026)
- [Privacy-Preserving AI Verification via Minimal Information Disclosure](https://mlsanity.com/paper/2608.02774/) (2026)
- [The Closing Window: How Governments Could Lose Their Ability to Restrain Advanced AI](https://mlsanity.com/paper/2608.05173/) (2026)
- [Adversarial Entropy Inflation Against Gumbel-Based Inference Verification](https://mlsanity.com/paper/2608.23375/) (2026)

## Abstract (arXiv)

As large AI models become increasingly valuable assets, the risk of model weight exfiltration from inference servers grows accordingly. An attacker controlling an inference server may exfiltrate model weights by hiding them within ordinary model responses, a strategy known as steganography. This work investigates how to verify LLM model inference to defend against such attacks and, more broadly, to detect anomalous or buggy behavior during inference. We formalize model weight exfiltration as a security game, propose a verification framework that can provably mitigate steganographic exfiltration, and specify the trust assumptions associated with our scheme. To enable verification, we characterize valid sources of non-determinism in large language model inference and introduce two practical estimators for them. We evaluate our detection framework on several open-weight models ranging from 3B to 30B parameters. On MOE-Qwen-30B, our detector reduces exfiltratable information to <0.5% with false-positive rate of <0.01%, corresponding to a >200x slowdown for adversaries. Overall, this work further establishes a foundation for defending against model weight exfiltration and demonstrates that strong protection can be achieved with minimal additional cost to inference providers. Our code is made public at: https://github.com/RoyRin/inference_verification_for_model_weight_exfiltration .