# Enhancing the De-identification of Personally Identifiable Information in Educational Data

arXiv: https://arxiv.org/abs/2501.09765 · HTML: https://mlsanity.com/paper/2501.09765/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Making Large Language Models Better Reasoners with Step-Aware Verifier](https://mlsanity.com/paper/2206.02336/) (2022)
- [From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference](https://mlsanity.com/paper/2310.03003/) (2023)
- [LAMOL: LAnguage MOdeling for Lifelong Language Learning](https://mlsanity.com/paper/1909.03329/) (2019)
- [Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation](https://mlsanity.com/paper/2305.16938/) (2023)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [C-sanitized: a privacy model for document redaction and sanitization](https://mlsanity.com/paper/1406.4285/) (2014)
- [Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubs](https://mlsanity.com/paper/2409.09368/) (2024)
- [Can We Count on LLMs? The Fixed-Effect Fallacy and Claims of GPT-4 Capabilities](https://mlsanity.com/paper/2409.07638/) (2024)
- [How Can I Improve? Using GPT to Highlight the Desired and Undesired Parts of Open-ended Responses](https://mlsanity.com/paper/2405.00291/) (2024)
- [Behind the Mask: Demographic bias in name detection for PII masking](https://mlsanity.com/paper/2205.04505/) (2022)
- [Few-shot Fine-tuning is All You Need for Source-free Domain Adaptation](https://mlsanity.com/paper/2304.00792/) (2023)
- [A Study of Optimizations for Fine-tuning Large Language Models](https://mlsanity.com/paper/2406.02290/) (2024)
- [A Comprehensive Study of Gender Bias in Chemical Named Entity Recognition Models](https://mlsanity.com/paper/2212.12799/) (2022)
- [Survey on Privacy-Preserving Techniques for Data Publishing](https://mlsanity.com/paper/2201.08120/) (2022)
- [BRATsynthetic: Text De-identification using a Markov Chain Replacement Strategy for Surrogate Personal Identifying Information](https://mlsanity.com/paper/2210.16125/) (2022)

## Built on by (most cited first, 7 of 7)

- [Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference](https://mlsanity.com/paper/2508.08438/) (2025)
- [An Evaluation Study of Hybrid Methods for Multilingual PII Detection](https://mlsanity.com/paper/2510.07551/) (2025)
- [An Agentic Workflow for Detecting Personally Identifiable Information in Crash Narratives](https://mlsanity.com/paper/2604.15369/) (2026)
- [Utility-Preserving De-Identification for Math Tutoring: Investigating Numeric Ambiguity in the MathEd-PII Benchmark Dataset](https://mlsanity.com/paper/2602.16571/) (2026)
- [Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations](https://mlsanity.com/paper/2512.00556/) (2025)
- [Towards Contextual Sensitive Data Detection](https://mlsanity.com/paper/2512.04120/) (2025)
- [Redact or Keep? A Fully Local AI Cascade for Educational Dialogue De-Identification](https://mlsanity.com/paper/2606.18372/) (2026)

## Abstract (arXiv)

Protecting Personally Identifiable Information (PII), such as names, is a critical requirement in learning technologies to safeguard student and teacher privacy and maintain trust. Accurate PII detection is an essential step toward anonymizing sensitive information while preserving the utility of educational data. Motivated by recent advancements in artificial intelligence, our study investigates the GPT-4o-mini model as a cost-effective and efficient solution for PII detection tasks. We explore both prompting and fine-tuning approaches and compare GPT-4o-mini's performance against established frameworks, including Microsoft Presidio and Azure AI Language. Our evaluation on two public datasets, CRAPII and TSCC, demonstrates that the fine-tuned GPT-4o-mini model achieves superior performance, with a recall of 0.9589 on CRAPII. Additionally, fine-tuned GPT-4o-mini significantly improves precision scores (a threefold increase) while reducing computational costs to nearly one-tenth of those associated with Azure AI Language. Furthermore, our bias analysis reveals that the fine-tuned GPT-4o-mini model consistently delivers accurate results across diverse cultural backgrounds and genders. The generalizability analysis using the TSCC dataset further highlights its robustness, achieving a recall of 0.9895 with minimal additional training data from TSCC. These results emphasize the potential of fine-tuned GPT-4o-mini as an accurate and cost-effective tool for PII detection in educational data. It offers robust privacy protection while preserving the data's utility for research and pedagogical analysis. Our code is available on GitHub: https://github.com/AnonJD/PrivacyAI