mlsanityData to 14 Oct 2026

Towards Selection of Large Multimodal Models as Engines for Burned-in Protected Health Information Detection in Medical Images

arXiv 2511.02014 ↗First posted 3 Nov 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The detection of Protected Health Information (PHI) in medical imaging is critical for safeguarding patient privacy and ensuring compliance with regulatory frameworks. Traditional detection methodologies predominantly utilize Optical Character Recognition (OCR) models in conjunction with named entity recognition. However, recent advancements in Large Multimodal Model (LMM) present new opportunities for enhanced text extraction and semantic analysis. In this study, we systematically benchmark three prominent closed and open-sourced LMMs, namely GPT-4o, Gemini 2.5 Flash, and Qwen 2.5 7B, utilizing two distinct pipeline configurations: one dedicated to text analysis alone and another integrating both OCR and semantic analysis. Our results indicate that LMM exhibits superior OCR efficacy (WER: 0.03-0.05, CER: 0.02-0.03) compared to conventional models like EasyOCR. However, this improvement in OCR performance does not consistently correlate with enhanced overall PHI detection accuracy. The strongest performance gains are observed on test cases with complex imprint patterns. In scenarios where text regions are well readable with sufficient contrast, and strong LMMs are employed for text analysis after OCR, different pipeline configurations yield similar results. Furthermore, we provide empirically grounded recommendations for LMM selection tailored to specific operational constraints and propose a deployment strategy that leverages scalable and modular infrastructure.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 21 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

10 of 21 references.

References · 10

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  5. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 279
  6. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  7. 2024De-Identification of Medical Imaging Data: A Comprehensive Tool for Ensuring Patient Privacycited by 7
  8. 2025Exploring AI-based System Design for Pixel-level Protected Health Information Detection in Medical Imagescited by 1
  9. 2025DICOM De-Identification via Hybrid AI and Rule-Based Framework for Scalable, Uncertainty-Aware Redactioncited by 1
  10. 2025Medical Image De-Identification Resources: Synthetic DICOM Data and Tools for Validationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Compact Multimodal Language Models as Robust OCR Alternatives for Noisy Textual Clinical Reports ↗
  2. 2026ClinOCR-Bench: A Comprehensive Clinical Scanned Document Dataset for Optical Character Recognition Model Evaluation
  3. 2025Vision Token Masking Alone Cannot Prevent PHI Leakage in Medical Document OCR: A Systematic Evaluation
  4. 2024ChatSchema: A pipeline of extracting structured information with Large Multimodal Models based on schema
  5. 2026Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection
  6. 2025A Multimodal Pipeline for Clinical Data Extraction: Applying Vision-Language Models to Scans of Transfusion Reaction Reports
  7. 2026Vision Language Model Helps Private Information De-Identification in Vision Data
  8. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning