# Doc-CoB: Enhancing Document Understanding with Visual Chain-of-Boxes Reasoning

arXiv: https://arxiv.org/abs/2505.18603 · HTML: https://mlsanity.com/paper/2505.18603/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 34 of its 44 references that are in the map.
- First posted on arXiv 24 May 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 34)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021)
- [mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding](https://mlsanity.com/paper/2403.12895/) (2024)
- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2503.12605/) (2025)
- [FeTaQA: Free-form Table Question Answering](https://mlsanity.com/paper/2104.00369/) (2021)
- [mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding](https://mlsanity.com/paper/2307.02499/) (2023)
- [Structured Chain-of-Thought Prompting for Code Generation](https://mlsanity.com/paper/2305.06599/) (2023)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness](https://mlsanity.com/paper/2412.11664/) (2024)
- [Hierarchical multimodal transformers for Multi-Page DocVQA](https://mlsanity.com/paper/2212.05935/) (2022)
- [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024)
- [Document AI: Benchmarks, Models and Applications](https://mlsanity.com/paper/2111.08609/) (2021)
- [Focus Anywhere for Fine-grained Multi-page Document Understanding](https://mlsanity.com/paper/2405.14295/) (2024)
- [T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answering](https://mlsanity.com/paper/2305.03453/) (2023)
- [ICDAR 2019 Competition on Scene Text Visual Question Answering](https://mlsanity.com/paper/1907.00490/) (2019)
- [GeoLayoutLM: Geometric Pre-training for Visual Information Extraction](https://mlsanity.com/paper/2304.10759/) (2023)
- [TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding](https://mlsanity.com/paper/2404.09797/) (2024)
- [Progressive Visual Prompt Learning with Contrastive Feature Re-formation](https://mlsanity.com/paper/2304.08386/) (2023)
- [VRDU: A Benchmark for Visually-rich Document Understanding](https://mlsanity.com/paper/2211.15421/) (2022)
- [Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering](https://mlsanity.com/paper/2306.00526/) (2023)
- [Multimodal Language Models See Better When They Look Shallower](https://mlsanity.com/paper/2504.21447/) (2025)
- [Token-level Correlation-guided Compression for Efficient Multimodal Document Understanding](https://mlsanity.com/paper/2407.14439/) (2024)

## Built on by (most cited first, 1 of 1)

- [DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding](https://mlsanity.com/paper/2508.07313/) (2025)

## Abstract (arXiv)

Document understanding aims to perform question answering and information extraction over document images, where the visual content is highly information-dense and most queries rely on only a few relevant layout regions. However, existing methods either adopt a one-pass strategy that implicitly assumes all layouts are equally important, or focus excessively on small regions at the cost of losing critical layout information. To address these limitations, we introduce Doc-CoB (Chain-of-Boxes), a simple-yet-effective framework that integrates coarse-to-fine layout-aware visual reasoning into multimodal large language models. Instead of directly zooming into small regions, Doc-CoB progressively focuses on query-relevant layouts while preserving global document information. Specifically, it first selects key layout boxes and then focuses on them for further understanding with visual prompting. To support this paradigm, we introduce two reasoning tasks for box recognition and box reasoning, with an automatic pipeline that constructs 249k training samples with intermediate visual supervision. Experiments on seven benchmarks with four popular models show that Doc-CoB significantly improves performance, demonstrating its effectiveness and wide applicability. The code and the data are available at https://github.com/Doc-CoB/Doc-CoB.