# Improving MLLM Historical Record Extraction with Test-Time Image

arXiv: https://arxiv.org/abs/2509.09722 · HTML: https://mlsanity.com/paper/2509.09722/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?](https://mlsanity.com/paper/2503.24235/) (2025)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Three things everyone should know about Vision Transformers](https://mlsanity.com/paper/2203.09795/) (2022)
- [End-to-end Document Recognition and Understanding with Dessurt](https://mlsanity.com/paper/2203.16618/) (2022)
- [Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More](https://mlsanity.com/paper/2502.03738/) (2025)
- [Confidence in the Reasoning of Large Language Models](https://mlsanity.com/paper/2412.15296/) (2024)
- [Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR](https://mlsanity.com/paper/2504.11101/) (2025)
- [Benchmarking Large Language Models for Handwritten Text Recognition](https://mlsanity.com/paper/2503.15195/) (2025)
- [Early evidence of how LLMs outperform traditional systems on OCR/HTR tasks for historical records](https://mlsanity.com/paper/2501.11623/) (2025)
- [Robustness of Structured Data Extraction from In-plane Rotated Documents using Multi-Modal Large Language Models (LLM)](https://mlsanity.com/paper/2406.10295/) (2024)
- [Test-time generative augmentation for medical image segmentation](https://mlsanity.com/paper/2406.17608/) (2024)
- [Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models](https://mlsanity.com/paper/2504.03440/) (2025)
- [UniViTAR: Unified Vision Transformer with Native Resolution](https://mlsanity.com/paper/2504.01792/) (2025)
- [Confidence-Aware Document OCR Error Detection](https://mlsanity.com/paper/2409.04117/) (2024)

## Abstract (arXiv)

We present a novel ensemble framework that stabilizes LLM based text extraction from noisy historical documents. We transcribe multiple augmented variants of each image with Gemini 2.0 Flash and fuse these outputs with a custom Needleman Wunsch style aligner that yields both a consensus transcription and a confidence score. We present a new dataset of 622 Pennsylvania death records, and demonstrate our method improves transcription accuracy by 4 percentage points relative to a single shot baseline. We find that padding and blurring are the most useful for improving accuracy, while grid warp perturbations are best for separating high and low confidence cases. The approach is simple, scalable, and immediately deployable to other document collections and transcription models.