# Exploring AI-based System Design for Pixel-level Protected Health Information Detection in Medical Images

arXiv: https://arxiv.org/abs/2501.09552 · HTML: https://mlsanity.com/paper/2501.09552/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Tasks](https://mlsanity.com/paper/2310.04270/) (2023)
- [De-Identification of Medical Imaging Data: A Comprehensive Tool for Ensuring Patient Privacy](https://mlsanity.com/paper/2410.12402/) (2024)
- [Medical Image Deidentification, Cleaning and Compression Using Pylogik](https://mlsanity.com/paper/2304.12322/) (2023)

## Built on by (most cited first, 1 of 1)

- [Towards Selection of Large Multimodal Models as Engines for Burned-in Protected Health Information Detection in Medical Images](https://mlsanity.com/paper/2511.02014/) (2025)

## Abstract (arXiv)

De-identification of medical images is a critical step to ensure privacy during data sharing in research and clinical settings. The initial step in this process involves detecting Protected Health Information (PHI), which can be found in image metadata or imprinted within image pixels. Despite the importance of such systems, there has been limited evaluation of existing AI-based solutions, creating barriers to the development of reliable and robust tools. In this study, we present an AI-based pipeline for PHI detection, comprising three key modules: text detection, text extraction, and text analysis. We benchmark three models - YOLOv11, EasyOCR, and GPT-4o - across different setups corresponding to these modules, evaluating their performance on two different datasets encompassing multiple imaging modalities and PHI categories. Our findings indicate that the optimal setup involves utilizing dedicated vision and language models for each module, which achieves a commendable balance in performance, latency, and cost associated with the usage of Large Language Models (LLMs). Additionally, we show that the application of LLMs not only involves identifying PHI content but also enhances OCR tasks and facilitates an end-to-end PHI detection pipeline, showcasing promising outcomes through our analysis.