# Operationalizing Document AI: A Microservice Architecture for OCR and LLM Pipelines in Production

arXiv: https://arxiv.org/abs/2605.18818 · HTML: https://mlsanity.com/paper/2605.18818/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [Nougat: Neural Optical Understanding for Academic Documents](https://mlsanity.com/paper/2308.13418/) (2023)
- [PP-OCR: A Practical Ultra Lightweight OCR System](https://mlsanity.com/paper/2009.09941/) (2020)
- [RF-DETR: Neural Architecture Search for Real-Time Detection Transformers](https://mlsanity.com/paper/2511.09554/) (2025)
- [Docling Technical Report](https://mlsanity.com/paper/2408.09869/) (2024)
- [DocVLM: Make Your VLM an Efficient Reader](https://mlsanity.com/paper/2412.08746/) (2024)
- [Hybrid OCR-LLM Framework for Enterprise-Scale Document Information Extraction Under Copy-heavy Task](https://mlsanity.com/paper/2510.10138/) (2025)
- [MMORE: Massive Multimodal Open RAG & Extraction](https://mlsanity.com/paper/2509.11937/) (2025)
- [A Hybrid Architecture for Multi-Stage Claim Document Understanding: Combining Vision-Language Models and Machine Learning for Real-Time Processing](https://mlsanity.com/paper/2601.01897/) (2026)
- [IDP Accelerator: Agentic Document Intelligence from Extraction to Compliance Validation](https://mlsanity.com/paper/2602.23481/) (2026)
- [OCR or Not? Rethinking Document Information Extraction in the MLLMs Era with Real-World Large-Scale Datasets](https://mlsanity.com/paper/2603.02789/) (2026)

## Built on by (most cited first, 1 of 1)

- [A Dual-Track Framework for Template-Constrained LaTeX Conversion](https://mlsanity.com/paper/2606.23107/) (2026)

## Abstract (arXiv)

Academic research tends to focus on new models for document understanding creating a wide gap in the literature between model definition and running models at production scale. To close that gap, we present a microservice architecture that encapsulates pipelines of multiple models for classification, optical character recognition (OCR), and large language model structured field extraction as well as our experience running this pipeline on thousands of multi-page documents per hour. We describe our primary design decisions, including a hybrid classification, separation of GPU-bound inference from CPU-bound orchestration, use of asynchronous processing for the many IO-bound operations in the pipeline, and an independent, horizontal scaling strategy. Using batch profiling, we identified two surprising qualitative findings that shape production deployments: OCR, not language-model parsing, dominates end-to-end latency, and the system saturates at a concurrency determined by shared GPU-inference capacity rather than worker count. Our goal is to provide practitioners with concrete architectural patterns for building document understanding systems that work beyond the benchmark; effectively operationalizing models in production.