# A large-scale dataset for end-to-end table recognition in the wild arXiv: https://arxiv.org/abs/2303.14884 ยท HTML: https://mlsanity.com/paper/2303.14884/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Builds on 13 of its 30 references that are in the map. - First posted on arXiv 27 Mar 2023. ## Builds on (most cited first, 13 of 13) - [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020) - [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018) - [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019) - [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019) - [PubTables-1M: Towards comprehensive table extraction from unstructured documents](https://mlsanity.com/paper/2110.00061/) (2021) - [TableBank: A Benchmark Dataset for Table Detection and Recognition](https://mlsanity.com/paper/1903.01949/) (2019) - [Complicated Table Structure Recognition](https://mlsanity.com/paper/1908.04729/) (2019) - [Parsing Table Structures in the Wild](https://mlsanity.com/paper/2109.02199/) (2021) - [PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML](https://mlsanity.com/paper/2105.01848/) (2021) - [Extracting Scientific Figures with Distantly Supervised Neural Networks](https://mlsanity.com/paper/1804.02445/) (2018) - [TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition](https://mlsanity.com/paper/2106.10598/) (2021) - [CDeC-Net: Composite Deformable Cascade Network for Table Detection in Document Images](https://mlsanity.com/paper/2008.10831/) (2020) - [TabLeX: A Benchmark Dataset for Structure and Content Information Extraction from Scientific Tables](https://mlsanity.com/paper/2105.06400/) (2021) ## Built on by (most cited first, 24 of 24) - [OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning](https://mlsanity.com/paper/2501.00321/) (2024) - [Detect Anything via Next Point Prediction](https://mlsanity.com/paper/2510.12798/) (2025) - [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024) - [NVIDIA Nemotron Nano V2 VL](https://mlsanity.com/paper/2511.03929/) (2025) - [Benchmarking Multimodal LLMs on Recognition and Understanding over Chemical Tables](https://mlsanity.com/paper/2506.11375/) (2025) - [DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Model](https://mlsanity.com/paper/2508.13238/) (2025) - [Vision as Unified Multimodal Generation](https://mlsanity.com/paper/2607.06560/) (2026) - [SPRINT: Script-agnostic Structure Recognition in Tables](https://mlsanity.com/paper/2503.11932/) (2025) - [Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition](https://mlsanity.com/paper/2404.13268/) (2024) - [SCORE: A Semantic Evaluation Framework for Generative Document Parsing](https://mlsanity.com/paper/2509.19345/) (2025) - [Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence](https://mlsanity.com/paper/2601.12318/) (2026) - [MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge](https://mlsanity.com/paper/2604.18164/) (2026) - [MonkeyOCRv2: A Visual-Text Foundation Model for Document AI](https://mlsanity.com/paper/2607.11562/) (2026) - [Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understanding](https://mlsanity.com/paper/2505.05446/) (2025) - [Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesis](https://mlsanity.com/paper/2511.12658/) (2025) - [StrucTab: A Structured Optimization Framework for Table Parsing](https://mlsanity.com/paper/2606.29905/) (2026) - [NVIDIA Nemotron Parse 1.1](https://mlsanity.com/paper/2511.20478/) (2025) - [TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignment](https://mlsanity.com/paper/2603.22819/) (2026) - [CISOL: An Open and Extensible Dataset for Table Structure Recognition in the Construction Industry](https://mlsanity.com/paper/2501.15469/) (2025) - [Hierarchical Modeling Approach to Fast and Accurate Table Recognition](https://mlsanity.com/paper/2512.21083/) (2025) - [TableNet A Large-Scale Table Dataset with LLM-Powered Autonomous](https://mlsanity.com/paper/2604.13041/) (2026) - [Benchmarking Large Vision-Language Models on CFMME: A Comprehensive Chinese Financial Multimodal Evaluation Dataset](https://mlsanity.com/paper/2605.29462/) (2026) - [Enhancing Table Structure Recognition via Bounding Box Guidance](https://mlsanity.com/paper/2609.08705/) (2026) - [Tables Decoded: DELTA for Structure, TARQA for Understanding](https://mlsanity.com/paper/2609.17458/) (2026) ## Abstract (arXiv) Table recognition (TR) is one of the research hotspots in pattern recognition, which aims to extract information from tables in an image. Common table recognition tasks include table detection (TD), table structure recognition (TSR) and table content recognition (TCR). TD is to locate tables in the image, TCR recognizes text content, and TSR recognizes spatial ogical structure. Currently, the end-to-end TR in real scenarios, accomplishing the three sub-tasks simultaneously, is yet an unexplored research area. One major factor that inhibits researchers is the lack of a benchmark dataset. To this end, we propose a new large-scale dataset named Table Recognition Set (TabRecSet) with diverse table forms sourcing from multiple scenarios in the wild, providing complete annotation dedicated to end-to-end TR research. It is the largest and first bi-lingual dataset for end-to-end TR, with 38.1K tables in which 20.4K are in English, and 17.7K are in Chinese. The samples have diverse forms, such as the border-complete and -incomplete table, regular and irregular table (rotated, distorted, etc.). The scenarios are multiple in the wild, varying from scanned to camera-taken images, documents to Excel tables, educational test papers to financial invoices. The annotations are complete, consisting of the table body spatial annotation, cell spatial logical annotation and text content for TD, TSR and TCR, respectively. The spatial annotation utilizes the polygon instead of the bounding box or quadrilateral adopted by most datasets. The polygon spatial annotation is more suitable for irregular tables that are common in wild scenarios. Additionally, we propose a visualized and interactive annotation tool named TableMe to improve the efficiency and quality of table annotation.