VLM-in-the-Loop: A Plug-In Quality Assurance Module for ECG Digitization Pipelines
Not on the map. Why?
Abstract (arXiv)
ECG digitization could unlock billions of archived clinical records, yet existing methods collapse on real-world images despite strong benchmark numbers. We introduce VLM-in-the-Loop, a plug-in quality assurance module that wraps any digitization backend with closed-loop VLM feedback via a standardized interface, requiring no modification to the underlying digitizer. The core mechanism is tool grounding: anchoring VLM assessment in quantitative evidence from domain-specific signal analysis tools. In a controlled ablation on 200 records with paired ground truth, tool grounding raises verdict consistency from 71% to 89% and doubles fidelity separation (DeltaPCC 0.03 rightarrow 0.08), with the effect replicating across three VLMs (Claude Opus~4, GPT-4o, Gemini~2.5 Pro), confirming a pattern-level rather than model-specific gain. Deployed across four backends, the module improves every one: 29.4% of borderline leads improved on our pipeline; 41.2% of failed limb leads recovered on ECG-Digitiser; valid leads per image doubled on Open-ECG-Digitizer (2.5 rightarrow 5.8). On 428 real clinical HCM images, the integrated system reaches 98.0% Excellent quality. Both the plug-in architecture and tool-grounding mechanism are domain-parametric, suggesting broader applicability wherever quality criteria are objectively measurable.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 30 references that are in the map. First posted on arXiv 1 Apr 2026.
Builds on
19 of 30 references.
References · 19
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
- 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
- 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
- 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
- 2023Generative Judge for Evaluating Alignmentcited by 163
- 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 142
- 2024Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluationcited by 123
- 2024OffsetBias: Leveraging Debiased Data for Tuning Evaluatorscited by 94
- 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
- 2024ECG-Chat: A Large ECG-Language Model for Cardiac Disease Diagnosiscited by 37
7 more
- 2023ECG-Image-Kit: A Synthetic Image Generation Toolbox to Facilitate Deep Learning-Based Electrocardiogram Digitizationcited by 20
- 2025Self-Improving VLM Judges Without Human Annotationscited by 7
- 2024Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Printoutscited by 5
- 2025Agentic Systems in Radiology: Design, Applications, Evaluation, and Challengescited by 5
- 2025Digitizing Paper ECGs at Scale: An Open-Source Algorithm for Clinical Researchcited by 3
- 2026Improving the Safety and Trustworthiness of Medical AI via Multi-Agent Evaluation Loopscited by 3
- 2025AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessmentcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretation
- 2026ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
- 2026Failures Reveal What Metrics Miss: An Evidence-Driven Agent for Recursive Refinement of ECG Classifiers
- 2026SHIFT-M3: Pre-fusion Alignment-based Consistency Screening for Multimodal ECG Record Integrity
- 2025GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images
- 2026CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations
- 2026Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis
- 2026ECGQuest: Benchmarking and Fine-Tuning Language Models for Electrocardiography