Improving MLLM Historical Record Extraction with Test-Time Image
Not on the map. Why?
Abstract (arXiv)
We present a novel ensemble framework that stabilizes LLM based text extraction from noisy historical documents. We transcribe multiple augmented variants of each image with Gemini 2.0 Flash and fuse these outputs with a custom Needleman Wunsch style aligner that yields both a consensus transcription and a confidence score. We present a new dataset of 622 Pennsylvania death records, and demonstrate our method improves transcription accuracy by 4 percentage points relative to a single shot baseline. We find that padding and blurring are the most useful for improving accuracy, while grid warp perturbations are best for separating high and low confidence cases. The approach is simple, scalable, and immediately deployable to other document collections and transcription models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 10 Sep 2025.
Builds on
20 of 32 references.
References · 20
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
- 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
- 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
- 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
- 2021OCR-free Document Understanding Transformercited by 407
- 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
- 2023Large Language Models for Generative Information Extraction: A Surveycited by 185
- 2022Three things everyone should know about Vision Transformerscited by 90
- 2022End-to-end Document Recognition and Understanding with Dessurtcited by 73
- 2025Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And Morecited by 28
- 2024Confidence in the Reasoning of Large Language Modelscited by 18
8 more
- 2025Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCRcited by 12
- 2025Benchmarking Large Language Models for Handwritten Text Recognitioncited by 11
- 2025Early evidence of how LLMs outperform traditional systems on OCR/HTR tasks for historical recordscited by 10
- 2024Robustness of Structured Data Extraction from In-plane Rotated Documents using Multi-Modal Large Language Models (LLM)cited by 7
- 2024Test-time generative augmentation for medical image segmentationcited by 6
- 2025Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Modelscited by 6
- 2025UniViTAR: Unified Vision Transformer with Native Resolutioncited by 3
- 2024Confidence-Aware Document OCR Error Detectioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Multimodal LLMs for OCR, OCR Post-Correction, and Named Entity Recognition in Historical Documents
- 2024Unlocking the Archives: Using Large Language Models to Transcribe Handwritten Historical Documents
- 2025CM1 -- A Dataset for Evaluating Few-Shot Information Extraction with Large Vision Language Models
- 2025Evaluating LLMs for Historical Document OCR: A Methodological Framework for Digital Humanities
- 2026Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction
- 2026From Pixels to Structure: Lightweight Vision-Language Models for Document OCR and Structured JSON Extraction
- 2026OCR or Not? Rethinking Document Information Extraction in the MLLMs Era with Real-World Large-Scale Datasets
- 2026Double Triangle Annotation: A Scalable Human-in-the-Loop Framework for High-Precision Historical Document Annotation