# VLM-in-the-Loop: A Plug-In Quality Assurance Module for ECG Digitization Pipelines

arXiv: https://arxiv.org/abs/2604.00396 · HTML: https://mlsanity.com/paper/2604.00396/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models](https://mlsanity.com/paper/2405.01535/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [Agent-as-a-Judge: Evaluate Agents with Agents](https://mlsanity.com/paper/2410.10934/) (2024)
- [Generative Judge for Evaluating Alignment](https://mlsanity.com/paper/2310.05470/) (2023)
- [Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences](https://mlsanity.com/paper/2404.12272/) (2024)
- [Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation](https://mlsanity.com/paper/2401.06591/) (2024)
- [OffsetBias: Leveraging Debiased Data for Tuning Evaluators](https://mlsanity.com/paper/2407.06551/) (2024)
- [Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation](https://mlsanity.com/paper/2407.10817/) (2024)
- [ECG-Chat: A Large ECG-Language Model for Cardiac Disease Diagnosis](https://mlsanity.com/paper/2408.08849/) (2024)
- [ECG-Image-Kit: A Synthetic Image Generation Toolbox to Facilitate Deep Learning-Based Electrocardiogram Digitization](https://mlsanity.com/paper/2307.01946/) (2023)
- [Self-Improving VLM Judges Without Human Annotations](https://mlsanity.com/paper/2512.05145/) (2025)
- [Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Printouts](https://mlsanity.com/paper/2410.14185/) (2024)
- [Agentic Systems in Radiology: Design, Applications, Evaluation, and Challenges](https://mlsanity.com/paper/2510.09404/) (2025)
- [Digitizing Paper ECGs at Scale: An Open-Source Algorithm for Clinical Research](https://mlsanity.com/paper/2510.19590/) (2025)
- [Improving the Safety and Trustworthiness of Medical AI via Multi-Agent Evaluation Loops](https://mlsanity.com/paper/2601.13268/) (2026)
- [AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment](https://mlsanity.com/paper/2509.26006/) (2025)

## Abstract (arXiv)

ECG digitization could unlock billions of archived clinical records, yet existing methods collapse on real-world images despite strong benchmark numbers. We introduce VLM-in-the-Loop, a plug-in quality assurance module that wraps any digitization backend with closed-loop VLM feedback via a standardized interface, requiring no modification to the underlying digitizer. The core mechanism is tool grounding: anchoring VLM assessment in quantitative evidence from domain-specific signal analysis tools. In a controlled ablation on 200 records with paired ground truth, tool grounding raises verdict consistency from 71% to 89% and doubles fidelity separation (DeltaPCC 0.03 rightarrow 0.08), with the effect replicating across three VLMs (Claude Opus~4, GPT-4o, Gemini~2.5 Pro), confirming a pattern-level rather than model-specific gain. Deployed across four backends, the module improves every one: 29.4% of borderline leads improved on our pipeline; 41.2% of failed limb leads recovered on ECG-Digitiser; valid leads per image doubled on Open-ECG-Digitizer (2.5 rightarrow 5.8). On 428 real clinical HCM images, the integrated system reaches 98.0% Excellent quality. Both the plug-in architecture and tool-grounding mechanism are domain-parametric, suggesting broader applicability wherever quality criteria are objectively measurable.