mlsanityData to 7 Oct 2026

Automating Governing Knowledge Commons and Contextual Integrity (GKC-CI) Privacy Policy Annotations with Large Language Models

arXiv 2311.02192 ↗First posted 3 Nov 2023cs.CY cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Identifying contextual integrity (CI) and governing knowledge commons (GKC) parameters in privacy policy texts can facilitate normative privacy analysis. However, GKC-CI annotation has heretofore required manual or crowdsourced effort. This paper demonstrates that high-accuracy GKC-CI parameter annotation of privacy policies can be performed automatically using large language models. We fine-tune 50 open-source and proprietary models on 21,588 ground truth GKC-CI annotations from 16 privacy policies. Our best performing model has an accuracy of 90.65%, which is comparable to the accuracy of experts on the same task. We apply our best performing model to 456 privacy policies from a variety of online services, demonstrating the effectiveness of scaling GKC-CI annotation for privacy policy exploration and analysis. We publicly release our model training code, training and testing data, an annotation visualizer, and all annotated policies for future GKC-CI research.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 30 of its 88 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

30 of 88 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2020Longformer: The Long-Document Transformercited by 3,362
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
18 more
  1. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  4. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  5. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  6. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  7. 2021How Many Data Points is a Prompt Worth?cited by 197
  8. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
  9. 2019Question Answering for Privacy Policies: Combining Computational and Legal Perspectivescited by 65
  10. 2018Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulationcited by 61
  11. 2018Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learningcited by 50
  12. 2020Privacy Policies over Time: Curation and Analysis of a Million-Document Datasetcited by 28
  13. 2015Android Permissions Remystified: A Field Study on Contextual Integritycited by 15
  14. 2023Reformulating Domain Adaptation of Large Language Models as Adapt-Retrieve-Revise: A Case Study on Chinese Legal Domaincited by 12
  15. 2021Brittle AI, Causal Confusion, and Bad Mental Models: Challenges and Successes in the XAI Programcited by 11
  16. 2018Discovering Smart Home Internet of Things Privacy Norms Using Contextual Integritycited by 10
  17. 2019Evaluating the Contextual Integrity of Privacy Regulation: Parents' IoT Toy Privacy Norms Versus COPPAcited by 4
  18. 2022Privacy Policies Across the Ages: Content and Readability of Privacy Policies 1996--2021cited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026PrivPRISM: Automatically Detecting Discrepancies Between Google Play Data Safety Declarations and Developer Privacy Policiescited by 1
  2. 2026PrivSTRUCT: Untangling Data Purpose Compliance of Privacy Policies in Google Play Storecited by 0
  3. 2026A Systematic Evaluation of Traditional Privacy Policy Analysis Tools Against LLMscited by 0