# The Right Information Extraction Pipeline Depends on the Document: Accuracy-Energy Trade-offs for Small, Local Models

arXiv: https://arxiv.org/abs/2609.31341 · HTML: https://mlsanity.com/paper/2609.31341/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Builds on (most cited first, 24 of 24)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Chargrid: Towards Understanding 2D Documents](https://mlsanity.com/paper/1809.08799/) (2018)
- [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024)
- [DeepSeek-OCR 2: Visual Causal Flow](https://mlsanity.com/paper/2601.20552/) (2026)
- [BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding](https://mlsanity.com/paper/1909.04948/) (2019)
- [Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion](https://mlsanity.com/paper/2501.17887/) (2025)
- [Energy Considerations of Large Language Model Inference and Efficiency Optimizations](https://mlsanity.com/paper/2504.17674/) (2025)
- [VRDU: A Benchmark for Visually-rich Document Understanding](https://mlsanity.com/paper/2211.15421/) (2022)
- [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024)
- [Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements](https://mlsanity.com/paper/2509.22092/) (2025)
- [Problem Solved? Information Extraction Design Space for Layout-Rich Documents using LLMs](https://mlsanity.com/paper/2502.18179/) (2025)
- [Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures](https://mlsanity.com/paper/2604.09048/) (2026)
- [Bench360: Benchmarking Local LLM Inference from 360 Degrees](https://mlsanity.com/paper/2511.16682/) (2025)
- [Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use](https://mlsanity.com/paper/2601.22362/) (2026)
- [Arctic-TILT. Business Document Understanding at Sub-Billion Scale](https://mlsanity.com/paper/2408.04632/) (2024)
- [Benchmarking Energy Efficiency of Large Language Models Using vLLM](https://mlsanity.com/paper/2509.08867/) (2025)
- [AgenticIE: An Adaptive Agent for Information Extraction from Complex Regulatory Documents](https://mlsanity.com/paper/2509.11773/) (2025)
- [Cooling Matters: Benchmarking Large Language Models and Vision-Language Models on Liquid-Cooled Versus Air-Cooled H100 GPU Systems](https://mlsanity.com/paper/2507.16781/) (2025)
- [WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs](https://mlsanity.com/paper/2607.02391/) (2026)

## Abstract (arXiv)

Whether an information extraction pipeline should process page images or parsed text depends on the document, and the answer flips across the layout spectrum. We study this trade-off under a constraint that rules out (closed) cloud services: privacy-sensitive documents processed on-premise by small (≤ 8B parameter) text-only and vision--language models, evaluated on both accuracy and energy over a design space spanning input representation, model family, and inference configuration. Benchmarking on the near-plain-text Kleister-NDA contracts and the layout-rich VRDU forms, we find that batching is the dominant energy lever, cutting energy per page by 38-85% at no cost in accuracy, while FP8 quantization saves 27-32% when requests are served one at a time but less than 1mWh per page (9-19%) once batching is applied. Preprocessing dominates what remains: neural OCR costs 17× more energy per page than classical OCR and never reaches the Pareto frontier. Which representation wins flips with the type of document: vision--language models on layout-rich documents and small text-only models with a cheap parser on near-plain text, where they are both more accurate and cheaper than any vision--language configuration. Our work yields concrete guidelines for energy-efficient, privacy-compliant local information extraction.