mlsanityData to 14 Oct 2026

TRACE: TRansformer-based Attribution using Contrastive Embeddings in LLMs

arXiv 2407.04981 ↗First posted 6 Jul 2024cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

The rapid evolution of large language models (LLMs) represents a substantial leap forward in natural language understanding and generation. However, alongside these advancements come significant challenges related to the accountability and transparency of LLM responses. Reliable source attribution is essential to adhering to stringent legal and regulatory standards, including those set forth by the General Data Protection Regulation. Despite the well-established methods in source attribution within the computer vision domain, the application of robust attribution frameworks to natural language processing remains underexplored. To bridge this gap, we propose a novel and versatile TRansformer-based Attribution framework using Contrastive Embeddings called TRACE that, in particular, exploits contrastive learning for source attribution. We perform an extensive empirical evaluation to demonstrate the performance and efficiency of TRACE in various settings and show that TRACE significantly improves the ability to attribute sources accurately, making it a valuable tool for enhancing the reliability and trustworthiness of LLMs.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

33 of 43 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
21 more
  1. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  2. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  3. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  4. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  5. 2018Universal Sentence Encodercited by 489
  6. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 432
  7. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  8. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  9. 2021Parametric Contrastive Learningcited by 217
  10. 2020Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Datacited by 201
  11. 2018Do GANs leave artificial fingerprints?cited by 190
  12. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  13. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  14. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 175
  15. 2020Membership Leakage in Label-Only Exposurescited by 168
  16. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 147
  17. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  18. 2024Proving membership in LLM pretraining data via data watermarkscited by 40
  19. 2023Source Attribution for Large Language Model-Generated Datacited by 11
  20. 2019Global Vectors for Node Representationscited by 10
  21. 2023Identifying Informational Sources in News Articlescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Reliable End-to-End Material Information Extraction from the Literature with Source-Tracked Multi-Stage Large Language Modelscited by 0
  2. 2026Interpretable Predictability-Based AI Text Detection: A Replication Studycited by 0
  3. 2026DataDignity: Training Data Attribution for Large Language Modelscited by 0
  4. 2026Train, Retrieve, or Both? A Four-Arm Head-to-Head for Correct Statutory Citation on the Ontario Residential Tenancies Actcited by 0