# Neural Collaborative Graph Machines for Table Structure Recognition

arXiv: https://arxiv.org/abs/2111.13359 · HTML: https://mlsanity.com/paper/2111.13359/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [ERNIE: Enhanced Representation through Knowledge Integration](https://mlsanity.com/paper/1904.09223/) (2019)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [Adaptively Sparse Transformers](https://mlsanity.com/paper/1909.00015/) (2019)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [SelfDoc: Self-Supervised Document Representation Learning](https://mlsanity.com/paper/2106.03331/) (2021)
- [TableBank: A Benchmark Dataset for Table Detection and Recognition](https://mlsanity.com/paper/1903.01949/) (2019)
- [Complicated Table Structure Recognition](https://mlsanity.com/paper/1908.04729/) (2019)
- [TableNet: Deep Learning model for end-to-end Table detection and Tabular data extraction from Scanned Document Images](https://mlsanity.com/paper/2001.01469/) (2020)
- [Table Structure Recognition using Top-Down and Bottom-Up Cues](https://mlsanity.com/paper/2010.04565/) (2020)
- [LGPMA: Complicated Table Structure Recognition with Local and Global Pyramid Mask Alignment](https://mlsanity.com/paper/2105.06224/) (2021)
- [Parsing Table Structures in the Wild](https://mlsanity.com/paper/2109.02199/) (2021)
- [Table Structure Extraction with Bi-directional Gated Recurrent Unit Networks](https://mlsanity.com/paper/2001.02501/) (2020)
- [Sparse and Constrained Attention for Neural Machine Translation](https://mlsanity.com/paper/1805.08241/) (2018)

## Built on by (most cited first, 30 of 30)

- [TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy](https://mlsanity.com/paper/2406.01326/) (2024)
- [Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation](https://mlsanity.com/paper/2310.16809/) (2023)
- [Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modeling](https://mlsanity.com/paper/2303.06949/) (2023)
- [TRUST: An Accurate and End-to-End Table structure Recognizer Using Splitting-based Transformers](https://mlsanity.com/paper/2208.14687/) (2022)
- [LORE: Logical Location Regression Network for Table Structure Recognition](https://mlsanity.com/paper/2303.03730/) (2023)
- [Doc2Graph: a Task Agnostic Document Understanding Framework based on Graph Neural Networks](https://mlsanity.com/paper/2208.11168/) (2022)
- [Aligning benchmark datasets for table structure recognition](https://mlsanity.com/paper/2303.00716/) (2023)
- [GridFormer: Towards Accurate Table Structure Recognition via Grid Prediction](https://mlsanity.com/paper/2309.14962/) (2023)
- [SEMv2: Table Separation Line Detection Based on Instance Segmentation](https://mlsanity.com/paper/2303.04384/) (2023)
- [Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer](https://mlsanity.com/paper/2303.11615/) (2023)
- [Relational Representation Learning in Visually-Rich Documents](https://mlsanity.com/paper/2205.02411/) (2022)
- [UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition](https://mlsanity.com/paper/2409.13148/) (2024)
- [TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism](https://mlsanity.com/paper/2501.11800/) (2025)
- [LORE++: Logical Location Regression Network for Table Structure Recognition with Pre-training](https://mlsanity.com/paper/2401.01522/) (2024)
- [SEMv3: A Fast and Robust Approach to Table Separation Line Detection](https://mlsanity.com/paper/2405.11862/) (2024)
- [Grab What You Need: Rethinking Complex Table Structure Recognition with Flexible Components Deliberation](https://mlsanity.com/paper/2303.09174/) (2023)
- [TRACE: Table Reconstruction Aligned to Corner and Edges](https://mlsanity.com/paper/2305.00630/) (2023)
- [Text Reading Order in Uncontrolled Conditions by Sparse Graph Segmentation](https://mlsanity.com/paper/2305.02577/) (2023)
- [TABLET: Table Structure Recognition using Encoder-only Transformers](https://mlsanity.com/paper/2506.07015/) (2025)
- [Synthesizing Realistic Data for Table Recognition](https://mlsanity.com/paper/2404.11100/) (2024)
- [DREAM: Document Reconstruction via End-to-end Autoregressive Model](https://mlsanity.com/paper/2507.05805/) (2025)
- [StrucTab: A Structured Optimization Framework for Table Parsing](https://mlsanity.com/paper/2606.29905/) (2026)
- [Rethinking Detection Based Table Structure Recognition for Visually Rich Document Images](https://mlsanity.com/paper/2312.00699/) (2023)
- [TableCenterNet: A one-stage network for table structure recognition](https://mlsanity.com/paper/2504.17522/) (2025)
- [Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction in Financial Documents at Scale](https://mlsanity.com/paper/2507.08865/) (2025)
- [TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignment](https://mlsanity.com/paper/2603.22819/) (2026)
- [SepFormer: Coarse-to-fine Separator Regression Network for Table Structure Recognition](https://mlsanity.com/paper/2506.21920/) (2025)
- [TableSeq: Unified Generation of Structure, Content, and Layout](https://mlsanity.com/paper/2604.16070/) (2026)
- [FastTab: A Fast Table Recognizer with a Tiny Recursive Module and 1D Transformers](https://mlsanity.com/paper/2605.22422/) (2026)
- [Enhancing Table Structure Recognition via Bounding Box Guidance](https://mlsanity.com/paper/2609.08705/) (2026)

## Abstract (arXiv)

Recently, table structure recognition has achieved impressive progress with the help of deep graph models. Most of them exploit single visual cues of tabular elements or simply combine visual cues with other modalities via early fusion to reason their graph relationships. However, neither early fusion nor individually reasoning in terms of multiple modalities can be appropriate for all varieties of table structures with great diversity. Instead, different modalities are expected to collaborate with each other in different patterns for different table cases. In the community, the importance of intra-inter modality interactions for table structure reasoning is still unexplored. In this paper, we define it as heterogeneous table structure recognition (Hetero-TSR) problem. With the aim of filling this gap, we present a novel Neural Collaborative Graph Machines (NCGM) equipped with stacked collaborative blocks, which alternatively extracts intra-modality context and models inter-modality interactions in a hierarchical way. It can represent the intra-inter modality relationships of tabular elements more robustly, which significantly improves the recognition performance. We also show that the proposed NCGM can modulate collaborative pattern of different modalities conditioned on the context of intra-modality cues, which is vital for diversified table cases. Experimental results on benchmarks demonstrate our proposed NCGM achieves state-of-the-art performance and beats other contemporary methods by a large margin especially under challenging scenarios.