mlsanityData to 14 Oct 2026

DataDignity: Training Data Attribution for Large Language Models

arXiv 2605.05687 ↗First posted 7 May 2026cs.AIcited by 0

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Auditing language-model outputs often requires more than judging correctness: an auditor may need to identify which source document most likely supports the knowledge expressed in a response. We study this as pinpoint provenance: given a prompt, a target-model response, and a candidate corpus, rank the documents that best support the response. We introduce FakeWiki, a controlled benchmark of 3,537 fabricated Wikipedia-style articles designed to preserve ground-truth provenance while weakening lexical shortcuts. FakeWiki includes QA probes, source-preserving paraphrases, retro-generated variants, hard anti-documents that remain topically similar while removing answer-critical facts, and five query conditions: clean prompting plus four jailbreak-inspired transformations. We evaluate seven retrieval baselines, a training-free activation-steering retrieval-fusion method, SteerFuse, and a supervised contrastive provenance ranker, ScoringModel. ScoringModel maps response and document features into a shared space and is trained with InfoNCE using in-batch, retrieval-mined, and anti-document negatives. Across nine open-weight instruction-tuned LLMs and five query conditions, ScoringModel improves mean Recall@10 from 35.0 for the strongest retrieval baseline to 52.2, without inference-time fusion, and wins 41/45 model-by-condition cells. SteerFuse is usually second-best despite requiring no supervised training, showing that activation-space evidence can efficiently complement text retrieval. On jailbreak-inspired transformed queries, ScoringModel improves Recall@10 by 15.7 points on average over the best baseline. Overall, our work shows that robust training data attribution requires evaluation settings that separate true answer support from topical or lexical resemblance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 22 of its 28 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

22 of 28 references.

References · 22

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  5. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2023Steering Language Models With Activation Engineeringcited by 922
  8. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  9. 2024Phi-4 Technical Reportcited by 711
  10. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  11. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  12. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
10 more
  1. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  2. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
  3. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 123
  4. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 81
  5. 2024Source-Aware Training Enables Knowledge Attribution in Language Modelscited by 30
  6. 2021Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updatescited by 27
  7. 2022Towards Tracing Factual Knowledge in Language Models Back to the Training Datacited by 21
  8. 2024Selection of LLM Fine-Tuning Data based on Orthogonal Rulescited by 12
  9. 2024TRACE: TRansformer-based Attribution using Contrastive Embeddings in LLMscited by 4
  10. 2024Fast Training Dataset Attribution via In-Context Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ProvenAI: Provenance-Native Traces of Evidence in Generated Answers
  2. 2025Data Provenance Auditing of Fine-Tuned Large Language Models with a Text-Preserving Technique
  3. 2026GenProve: Learning to Generate Text with Fine-Grained Provenance
  4. 2026TokenPrint: A Calibrated Token-Space Fingerprint for Language-Model Provenance
  5. 2026OriginBlame: Record- and Token-Level Data Provenance for AI Training Datasets
  6. 2023Trusted Source Alignment in Large Language Models
  7. 2024Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data
  8. 2024Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments