# PII-Bench: Evaluating Query-Aware Privacy Protection Systems

arXiv: https://arxiv.org/abs/2502.18545 · HTML: https://mlsanity.com/paper/2502.18545/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Builds on (most cited first, 13 of 13)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Analyzing Leakage of Personally Identifiable Information in Language Models](https://mlsanity.com/paper/2302.00539/) (2023)
- [Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue](https://mlsanity.com/paper/2308.03549/) (2023)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [De-identification of Patient Notes with Recurrent Neural Networks](https://mlsanity.com/paper/1606.03475/) (2016)
- [The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts](https://mlsanity.com/paper/2406.14318/) (2024)
- [Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Text](https://mlsanity.com/paper/2405.19831/) (2024)
- [DePrompt: Desensitization and Evaluation of Personal Identifiable Information in Large Language Model Prompts](https://mlsanity.com/paper/2408.08930/) (2024)
- [Benchmarking Advanced Text Anonymisation Methods: A Comparative Study on Novel and Traditional Approaches](https://mlsanity.com/paper/2404.14465/) (2024)

## Built on by (most cited first, 11 of 11)

- [A Survey on Privacy Risks and Protection in Large Language Models](https://mlsanity.com/paper/2505.01976/) (2025)
- [CAPID: Context-Aware PII Detection for Question-Answering Systems](https://mlsanity.com/paper/2602.10074/) (2026)
- [Minim: Privacy-Aware Minimal View for Agents via Trusted Local Sanitization](https://mlsanity.com/paper/2606.13949/) (2026)
- [Subject-level Inference for Realistic Text Anonymization Evaluation](https://mlsanity.com/paper/2604.21211/) (2026)
- [What to Remember, What to Reveal: Privacy-Aware Memory for Conversational Agents](https://mlsanity.com/paper/2608.16551/) (2026)
- [PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations](https://mlsanity.com/paper/2609.22200/) (2026)
- [Conversations Risk Detection LLMs in Financial Agents via Multi-Stage Generative Rollout](https://mlsanity.com/paper/2604.09056/) (2026)
- [Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs](https://mlsanity.com/paper/2605.06305/) (2026)
- [RedactionBench](https://mlsanity.com/paper/2606.18782/) (2026)
- [Unifying Conformal Language Tasks with In-Context Ensembles](https://mlsanity.com/paper/2609.03005/) (2026)
- [Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution](https://mlsanity.com/paper/2610.17816/) (2026)

## Abstract (arXiv)

The widespread adoption of Large Language Models (LLMs) has raised significant privacy concerns regarding the exposure of personally identifiable information (PII) in user prompts. To address this challenge, we propose a query-unrelated PII masking strategy and introduce PII-Bench, the first comprehensive evaluation framework for assessing privacy protection systems. PII-Bench comprises 2,842 test samples across 55 fine-grained PII categories, featuring diverse scenarios from single-subject descriptions to complex multi-party interactions. Each sample is carefully crafted with a user query, context description, and standard answer indicating query-relevant PII. Our empirical evaluation reveals that while current models perform adequately in basic PII detection, they show significant limitations in determining PII query relevance. Even state-of-the-art LLMs struggle with this task, particularly in handling complex multi-subject scenarios, indicating substantial room for improvement in achieving intelligent PII masking.