# Selective Token-Level Cryptographic Redaction for Privacy-Preserving Clinical Deployment of Large Language Models

arXiv: https://arxiv.org/abs/2606.03399 · HTML: https://mlsanity.com/paper/2606.03399/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 77 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://mlsanity.com/paper/2203.14371/) (2022)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [Beyond Memorization: Violating Privacy Via Inference with Large Language Models](https://mlsanity.com/paper/2310.07298/) (2023)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [Med42-v2: A Suite of Clinical LLMs](https://mlsanity.com/paper/2408.06142/) (2024)
- [Natural Language Understanding with Privacy-Preserving BERT](https://mlsanity.com/paper/2104.07504/) (2021)
- [Training Data Extraction From Pre-trained Language Models: A Survey](https://mlsanity.com/paper/2305.16157/) (2023)
- [Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches](https://mlsanity.com/paper/2211.16285/) (2022)
- [Differentially Private Decoding in Large Language Models](https://mlsanity.com/paper/2205.13621/) (2022)
- [Jailbreaking and Mitigation of Vulnerabilities in Large Language Models](https://mlsanity.com/paper/2410.15236/) (2024)
- [Protecting User Privacy in Remote Conversational Systems: A Privacy-Preserving framework based on text sanitization](https://mlsanity.com/paper/2306.08223/) (2023)
- [Encryption-Friendly LLM Architecture](https://mlsanity.com/paper/2410.02486/) (2024)
- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [Prεεmpt: Sanitizing Sensitive Prompts for LLMs](https://mlsanity.com/paper/2504.05147/) (2025)
- [SentinelLMs: Encrypted Input Adaptation and Fine-tuning of Language Models for Private and Secure Inference](https://mlsanity.com/paper/2312.17342/) (2023)
- [EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs](https://mlsanity.com/paper/2402.05868/) (2024)
- [A Survey of Secure Computation Using Trusted Execution Environments](https://mlsanity.com/paper/2302.12150/) (2023)
- [The Aloe Family Recipe for Open and Specialized Healthcare LLMs](https://mlsanity.com/paper/2505.04388/) (2025)
- [Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions](https://mlsanity.com/paper/2412.06113/) (2024)
- [Cryptanalysis via Machine Learning Based Information Theoretic Metrics](https://mlsanity.com/paper/2501.15076/) (2025)

## Abstract (arXiv)

While large language models (LLMs) are increasingly used for clinical applications, many existing pipelines require sending raw sensitive health information to remote servers for processing, which heightens the risk of privacy leakage. A natural approach to mitigate this risk is to encrypt the data before transmission. However, straightforward solutions such as encrypting the entire dataset introduce prohibitive computational, alignment, and communication overheads, rendering large-scale practical deployment infeasible. To preserve privacy while maintaining usability, we present Healthcare Encryption & Redaction via Adaptive Linguistic Decomposition (HERALD), a token-level cryptographic redaction framework designed to achieve this balance by encrypting only sensitive tokens while preserving the surrounding context for downstream model utility. HERALD combines medical named-entity recognizer (NER) with part-of-speech (POS) driven policies to select candidate tokens, performs targeted lemmatization to stabilize surface forms, and substitutes each protected token with a deterministic ciphertext wrapped in explicit delimiters. Notably, HERALD is model-agnostic and operates entirely on the client side, ensuring that sensitive content remains encrypted throughout storage, transmission, and processing without requiring changes to downstream models. We evaluated HERALD on both classification and medical question answering (MQA) tasks on public datasets. Across different tasks, experiments illustrate that fully secured baselines suffer significant utility loss, whereas HERALD consistently recovers performance close to plaintext. Overall, HERALD provides a novel utilization pipeline.