Perception, Layout, and Validation: Calibrated Confidence for Reliable Straight-Through Processing of Financial Documents
Not on the map. Why?
Abstract (arXiv)
Straight-through processing (STP) on extracted key-value fields from financial documents without human review requires a calibrated probability together with a bounded guarantee on the residual error of the auto-approved tier. The emergence of modern Vision Language Models (VLMs) provides an out-of-the-box capability for extracting the key-values, but their verbalized confidence signals are unreliable and weakly track field correctness. This paper introduces a decomposed confidence layer along three interpretable channels, including perception, layout, and validation. Together with a final conformal risk control, the score can be used for reliable STP of financial documents. The method is validated on three public datasets covering real invoices, synthetic invoices, and ad-buy forms, using two different VLM families (Qwen3.6-27B and Gemini-3.1-Flash-Lite). Our decomposed score consistently improves the separation of correct from incorrect extractions, substantially raising the AUROC from 0.54-0.74 for VLM verbalized signals to 0.90-0.99 with contributions from all three designed channels. Crucially for industrial deployment, this enables usable STP. The native VLM confidence signals could clear only 0.1%-7.0% of fields under risk control at a target error of <10%. In contrast, the proposed method auto-approves 49-72% of fields while holding the empirical error of the accepted tier at or below the target.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 25 references that are in the map. First posted on arXiv 17 Sep 2026.
Builds on
10 of 25 references.
References · 10
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
- 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
- 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
- 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 271
- 2023DocILE Benchmark for Document Information Localization and Extractioncited by 58
- 2022VRDU: A Benchmark for Visually-rich Document Understandingcited by 35
- 2023FATURA: A Multi-Layout Invoice Image Dataset for Document Analysis and Understandingcited by 4
- 2026Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extractioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Confidence Estimation for Financial Vision-Language Models in Chart and Document Understanding
- 2025FinCriticalED: A Visual Benchmark for Financial Fact-Level OCR
- 2026When Tables Go Crazy: Evaluating Multimodal Models on French Financial Documents
- 2026A Multistage Extraction Pipeline for Long Scanned Financial Documents: An Empirical Study in Industrial KYC Workflows
- 2026Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction
- 2025Multi-Stage Field Extraction of Financial Documents with OCR and Compact Vision-Language Models ↗
- 2026Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction
- 2026Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents