Refinement Provenance Inference: Detecting LLM-Refined Training Prompts from Model Behavior
Not on the map. Why?
Abstract (arXiv)
Instruction tuning increasingly relies on LLM-based prompt refinement, where prompts in the training corpus are selectively rewritten by an external refiner to improve clarity and instruction alignment. This motivates an instance-level audit problem: for a fine-tuned model and a training prompt-response pair, can we infer whether the model was trained on the original prompt or its LLM-refined version within a mixed corpus? This matters for dataset governance and dispute resolution when training data are contested. However, it is non-trivial in practice: refined and raw instances are interleaved in the training corpus with unknown, source-dependent mixture ratios, making it harder to develop provenance methods that generalize across models and training setups. In this paper, we formalize this audit task as Refinement Provenance Inference (RPI) and show that prompt refinement yields stable, detectable shifts in teacher-forced token distributions, even when semantic differences are not obvious. Building on this phenomenon, we propose RePro, a logit-based provenance framework that fuses teacher-forced likelihood features with logit-ranking signals. During training, RePro learns a transferable representation via shadow fine-tuning, and uses a lightweight linear head to infer provenance on unseen victims without training-data access. Empirically, RePro consistently attains strong performance and transfers well across refiners, suggesting that it exploits refiner-agnostic distribution shifts rather than rewrite-style artifacts.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 5 Jan 2026.
Builds on
33 of 43 references.
References · 33
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
- 2023LIMA: Less Is More for Alignmentcited by 1,080
- 2018Model Cards for Model Reportingcited by 1,068
- 2021Membership Inference Attacks From First Principlescited by 839
- 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
- 2023Whose Opinions Do Language Models Reflect?cited by 759
- 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
- 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
21 more
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
- 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
- 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
- 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
- 2023Robust Distortion-free Watermarks for Language Modelscited by 279
- 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 273
- 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
- 2023The False Promise of Imitating Proprietary LLMscited by 254
- 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
- 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 207
- 2023Auditing large language models: a three-layered approachcited by 135
- 2013Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifierscited by 128
- 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 127
- 2021Property Inference From Poisoningcited by 56
- 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Studycited by 37
- 2022Dataset Inference for Self-Supervised Modelscited by 36
- 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 17
- 2025FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuningcited by 7
- 2023Towards Robust and Accurate Visual Promptingcited by 5
- 2024Data Lineage Inference: Uncovering Privacy Vulnerabilities of Dataset Pruningcited by 5
- 2025LacaDM: A Latent Causal Diffusion Model for Multiobjective Reinforcement Learningcited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.