mlsanityData to 7 Oct 2026

PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English

arXiv 2212.10011 ↗First posted 20 Dec 2022cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Privacy policies provide individuals with information about their rights and how their personal information is handled. Natural language understanding (NLU) technologies can support individuals and practitioners to understand better privacy practices described in lengthy and complex documents. However, existing efforts that use NLU technologies are limited by processing the language in a way exclusive to a single task focusing on certain privacy practices. To this end, we introduce the Privacy Policy Language Understanding Evaluation (PLUE) benchmark, a multi-task benchmark for evaluating the privacy policy language understanding across various tasks. We also collect a large corpus of privacy policies to enable privacy policy domain-specific language model pre-training. We evaluate several generic pre-trained language models and continue pre-training them on the collected corpus. We demonstrate that domain-specific continual pre-training offers performance improvements across all tasks.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 54 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

23 of 54 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2018Deep contextualized word representationscited by 2,285
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  12. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
11 more
  1. 2019Publicly Available Clinical BERT Embeddingscited by 995
  2. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  3. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  4. 2022ELLE: Efficient Lifelong Pre-training for Emerging Datacited by 70
  5. 2019Question Answering for Privacy Policies: Combining Computational and Legal Perspectivescited by 65
  6. 2018Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learningcited by 50
  7. 2020PolicyQA: A Reading Comprehension Dataset for Privacy Policiescited by 31
  8. 2022CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domaincited by 31
  9. 2020Privacy Policies over Time: Curation and Analysis of a Million-Document Datasetcited by 28
  10. 2021Intent Classification and Slot Filling for Privacy Policiescited by 18
  11. 2022Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policiescited by 10

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 2.

5 papers.

  1. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  2. 2023Bringing order into the realm of Transformer-based language models for artificial intelligence and lawcited by 13
  3. 2024SOS! Soft Prompt Attack Against Open-Source Large Language Modelscited by 7
  4. 2025Benchmarking LLM Privacy Recognition for Social Robot Decision Makingcited by 3
  5. 2025A Multi-Agent Framework for Mitigating Dialect Biases in Privacy Policy Question-Answering Systemscited by 2