# Robust Table Detection and Structure Recognition from Heterogeneous Document Images

arXiv: https://arxiv.org/abs/2203.09056 · HTML: https://mlsanity.com/paper/2203.09056/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 88 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [Spatial As Deep: Spatial CNN for Traffic Scene Understanding](https://mlsanity.com/paper/1712.06080/) (2017)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [TableBank: A Benchmark Dataset for Table Detection and Recognition](https://mlsanity.com/paper/1903.01949/) (2019)
- [CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents](https://mlsanity.com/paper/2004.12629/) (2020)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [Complicated Table Structure Recognition](https://mlsanity.com/paper/1908.04729/) (2019)
- [TableNet: Deep Learning model for end-to-end Table detection and Tabular data extraction from Scanned Document Images](https://mlsanity.com/paper/2001.01469/) (2020)
- [Table Structure Recognition using Top-Down and Bottom-Up Cues](https://mlsanity.com/paper/2010.04565/) (2020)
- [LGPMA: Complicated Table Structure Recognition with Local and Global Pyramid Mask Alignment](https://mlsanity.com/paper/2105.06224/) (2021)
- [CBNet: A Novel Composite Backbone Network Architecture for Object Detection](https://mlsanity.com/paper/1909.03625/) (2019)
- [Split, embed and merge: An accurate table structure recognizer](https://mlsanity.com/paper/2107.05214/) (2021)
- [CDeC-Net: Composite Deformable Cascade Network for Table Detection in Document Images](https://mlsanity.com/paper/2008.10831/) (2020)
- [Table Structure Extraction with Bi-directional Gated Recurrent Unit Networks](https://mlsanity.com/paper/2001.02501/) (2020)
- [ReLaText: Exploiting Visual Relationships for Arbitrary-Shaped Scene Text Detection with Graph Convolutional Networks](https://mlsanity.com/paper/2003.06999/) (2020)

## Built on by (most cited first, 30 of 30)

- [Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modeling](https://mlsanity.com/paper/2303.06949/) (2023)
- [TSRFormer: Table Structure Recognition with Transformers](https://mlsanity.com/paper/2208.04921/) (2022)
- [Deep learning for table detection and structure recognition: A survey](https://mlsanity.com/paper/2211.08469/) (2022)
- [GridFormer: Towards Accurate Table Structure Recognition via Grid Prediction](https://mlsanity.com/paper/2309.14962/) (2023)
- [UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining](https://mlsanity.com/paper/2403.04822/) (2024)
- [SEMv2: Table Separation Line Detection Based on Instance Segmentation](https://mlsanity.com/paper/2303.04384/) (2023)
- [Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer](https://mlsanity.com/paper/2303.11615/) (2023)
- [UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition](https://mlsanity.com/paper/2409.13148/) (2024)
- [Enhancing Table Recognition with Vision LLMs: A Benchmark and Neighbor-Guided Toolchain Reasoner](https://mlsanity.com/paper/2412.20662/) (2024)
- [TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism](https://mlsanity.com/paper/2501.11800/) (2025)
- [Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language Models](https://mlsanity.com/paper/2509.17589/) (2025)
- [SEMv3: A Fast and Robust Approach to Table Separation Line Detection](https://mlsanity.com/paper/2405.11862/) (2024)
- [Grab What You Need: Rethinking Complex Table Structure Recognition with Flexible Components Deliberation](https://mlsanity.com/paper/2303.09174/) (2023)
- [TRACE: Table Reconstruction Aligned to Corner and Edges](https://mlsanity.com/paper/2305.00630/) (2023)
- [TABLET: Table Structure Recognition using Encoder-only Transformers](https://mlsanity.com/paper/2506.07015/) (2025)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [ExTTNet: A Deep Learning Algorithm for Extracting Table Texts from Invoice Images](https://mlsanity.com/paper/2402.02246/) (2024)
- [Rethinking Detection Based Table Structure Recognition for Visually Rich Document Images](https://mlsanity.com/paper/2312.00699/) (2023)
- [PdfTable: A Unified Toolkit for Deep Learning-Based Table Extraction](https://mlsanity.com/paper/2409.05125/) (2024)
- [TableCenterNet: A one-stage network for table structure recognition](https://mlsanity.com/paper/2504.17522/) (2025)
- [HCT-QA: A Benchmark for Question Answering on Human-Centric Tables](https://mlsanity.com/paper/2504.20047/) (2025)
- [TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignment](https://mlsanity.com/paper/2603.22819/) (2026)
- [InstructTable: Improving Table Structure Recognition Through Instructions](https://mlsanity.com/paper/2604.02880/) (2026)
- [Semantic Table Detection with LayoutLMv3](https://mlsanity.com/paper/2211.15504/) (2022)
- [TDeLTA: A Light-weight and Robust Table Detection Method based on Learning Text Arrangement](https://mlsanity.com/paper/2312.11043/) (2023)
- [RTHDet: Rotate Table Area and Head Detection in images](https://mlsanity.com/paper/2402.03315/) (2023)
- [SepFormer: Coarse-to-fine Separator Regression Network for Table Structure Recognition](https://mlsanity.com/paper/2506.21920/) (2025)
- [Rethinking the Pointer Loss in Table Structure Recognition: Geometry-Aware Pointer Loss for Spatial Locality](https://mlsanity.com/paper/2606.18721/) (2026)
- [ConRTF: Edge-Constrained Boundary Distribution Refinement for Realtime TransFormer Table Structure Recognition](https://mlsanity.com/paper/2607.00734/) (2026)
- [EMBLEM: Enhancing Multi-script Table Detection through Masking](https://mlsanity.com/paper/2609.08330/) (2026)

## Abstract (arXiv)

We introduce a new table detection and structure recognition approach named RobusTabNet to detect the boundaries of tables and reconstruct the cellular structure of each table from heterogeneous document images. For table detection, we propose to use CornerNet as a new region proposal network to generate higher quality table proposals for Faster R-CNN, which has significantly improved the localization accuracy of Faster R-CNN for table detection. Consequently, our table detection approach achieves state-of-the-art performance on three public table detection benchmarks, namely cTDaR TrackA, PubLayNet and IIIT-AR-13K, by only using a lightweight ResNet-18 backbone network. Furthermore, we propose a new split-and-merge based table structure recognition approach, in which a novel spatial CNN based separation line prediction module is proposed to split each detected table into a grid of cells, and a Grid CNN based cell merging module is applied to recover the spanning cells. As the spatial CNN module can effectively propagate contextual information across the whole table image, our table structure recognizer can robustly recognize tables with large blank spaces and geometrically distorted (even curved) tables. Thanks to these two techniques, our table structure recognition approach achieves state-of-the-art performance on three public benchmarks, including SciTSR, PubTabNet and cTDaR TrackB2-Modern. Moreover, we have further demonstrated the advantages of our approach in recognizing tables with complex structures, large blank spaces, as well as geometrically distorted or even curved shapes on a more challenging in-house dataset.