# Verifiable Fine-Tuning for LLMs: Zero-Knowledge Training Proofs Bound to Data Provenance and Policy

arXiv: https://arxiv.org/abs/2510.16830 · HTML: https://mlsanity.com/paper/2510.16830/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 65 references that are in the map.
- First posted on arXiv 19 Oct 2025.

## Builds on (most cited first, 29 of 29)

- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [On the Reliability of Watermarks for Large Language Models](https://mlsanity.com/paper/2306.04634/) (2023)
- [Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust](https://mlsanity.com/paper/2305.20030/) (2023)
- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [Watermark Stealing in Large Language Models](https://mlsanity.com/paper/2402.19361/) (2024)
- [Proof-of-Learning: Definitions and Practice](https://mlsanity.com/paper/2103.05633/) (2021)
- [On the Learnability of Watermarks for Language Models](https://mlsanity.com/paper/2312.04469/) (2023)
- [zkLLM: Zero Knowledge Proofs for Large Language Models](https://mlsanity.com/paper/2404.16109/) (2024)
- [Provably Robust Multi-bit Watermarking for AI-generated Text](https://mlsanity.com/paper/2401.16820/) (2024)
- [Verifiable and Provably Secure Machine Unlearning](https://mlsanity.com/paper/2210.09126/) (2022)
- [Tools for Verifying Neural Models' Training Data](https://mlsanity.com/paper/2307.00682/) (2023)
- [A Survey of Zero-Knowledge Proof Based Verifiable Machine Learning](https://mlsanity.com/paper/2502.18535/) (2025)
- [zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training](https://mlsanity.com/paper/2307.16273/) (2023)
- [Learning from Heterogeneity: Generalizing Dynamic Facial Expression Recognition via Distributionally Robust Optimization](https://mlsanity.com/paper/2507.15765/) (2025)
- [WiOpen: A Robust Wi-Fi-based Open-set Gesture Recognition Framework](https://mlsanity.com/paper/2402.00822/) (2024)
- [Model Provenance Testing for Large Language Models](https://mlsanity.com/paper/2502.00706/) (2025)
- [ReSup: Reliable Label Noise Suppression for Facial Expression Recognition](https://mlsanity.com/paper/2305.17895/) (2023)
- [Large Language Model Watermark Stealing With Mixed Integer Programming](https://mlsanity.com/paper/2405.19677/) (2024)
- [ezDPS: An Efficient and Zero-Knowledge Machine Learning Inference Pipeline](https://mlsanity.com/paper/2212.05428/) (2022)
- [Laminator: Verifiable ML Property Cards using Hardware-assisted Attestations](https://mlsanity.com/paper/2406.17548/) (2024)
- [FedQuad: Adaptive Layer-wise LoRA Deployment and Activation Quantization for Federated Fine-Tuning](https://mlsanity.com/paper/2506.01001/) (2025)
- [Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism](https://mlsanity.com/paper/2509.08342/) (2025)
- [Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization](https://mlsanity.com/paper/2501.12388/) (2024)
- [Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning](https://mlsanity.com/paper/2506.05977/) (2025)
- [Towards Communication-Efficient Decentralized Federated Graph Learning over Non-IID Data](https://mlsanity.com/paper/2509.08409/) (2025)
- [Attesting Distributional Properties of Training Data for Machine Learning](https://mlsanity.com/paper/2308.09552/) (2023)
- [Proof-of-Learning with Incentive Security](https://mlsanity.com/paper/2404.09005/) (2024)

## Built on by (most cited first, 2 of 2)

- [Committed SAE-Feature Traces for Audited-Session Substitution Detection in Hosted LLMs](https://mlsanity.com/paper/2604.18179/) (2026)
- [Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints](https://mlsanity.com/paper/2608.02154/) (2026)

## Abstract (arXiv)

Large language models are often adapted through parameter efficient fine tuning, but current release practices provide weak assurances about what data were used and how updates were computed. We present Verifiable Fine Tuning, a protocol and system that produces succinct zero knowledge proofs that a released model was obtained from a public initialization under a declared training program and an auditable dataset commitment. The approach combines five elements. First, commitments that bind data sources, preprocessing, licenses, and per epoch quota counters to a manifest. Second, a verifiable sampler that supports public replayable and private index hiding batch selection. Third, update circuits restricted to parameter efficient fine tuning that enforce AdamW style optimizer semantics and proof friendly approximations with explicit error budgets. Fourth, recursive aggregation that folds per step proofs into per epoch and end to end certificates with millisecond verification. Fifth, provenance binding and optional trusted execution property cards that attest code identity and constants. On English and bilingual instruction mixtures, the method maintains utility within tight budgets while achieving practical proof performance. Policy quotas are enforced with zero violations, and private sampling windows show no measurable index leakage. Federated experiments demonstrate that the system composes with probabilistic audits and bandwidth constraints. These results indicate that end to end verifiable fine tuning is feasible today for real parameter efficient pipelines, closing a critical trust gap for regulated and decentralized deployments.