# Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modeling

arXiv: https://arxiv.org/abs/2303.06949 · HTML: https://mlsanity.com/paper/2303.06949/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 59 references that are in the map.
- First posted on arXiv 13 Mar 2023.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [R-FCN: Object Detection via Region-based Fully Convolutional Networks](https://mlsanity.com/paper/1605.06409/) (2016)
- [Pix2seq: A Language Modeling Framework for Object Detection](https://mlsanity.com/paper/2109.10852/) (2021)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [Shape Robust Text Detection with Progressive Scale Expansion Network](https://mlsanity.com/paper/1903.12473/) (2019)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [PubTables-1M: Towards comprehensive table extraction from unstructured documents](https://mlsanity.com/paper/2110.00061/) (2021)
- [TableBank: A Benchmark Dataset for Table Detection and Recognition](https://mlsanity.com/paper/1903.01949/) (2019)
- [CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents](https://mlsanity.com/paper/2004.12629/) (2020)
- [TableFormer: Table Structure Understanding with Transformers](https://mlsanity.com/paper/2203.01017/) (2022)
- [Complicated Table Structure Recognition](https://mlsanity.com/paper/1908.04729/) (2019)
- [Image-to-Markup Generation with Coarse-to-Fine Attention](https://mlsanity.com/paper/1609.04938/) (2016)
- [TableNet: Deep Learning model for end-to-end Table detection and Tabular data extraction from Scanned Document Images](https://mlsanity.com/paper/2001.01469/) (2020)
- [MASTER: Multi-Aspect Non-local Network for Scene Text Recognition](https://mlsanity.com/paper/1910.02562/) (2019)
- [Table Structure Recognition using Top-Down and Bottom-Up Cues](https://mlsanity.com/paper/2010.04565/) (2020)
- [LGPMA: Complicated Table Structure Recognition with Local and Global Pyramid Mask Alignment](https://mlsanity.com/paper/2105.06224/) (2021)
- [Learning representations of irregular particle-detector geometry with distance-weighted graph networks](https://mlsanity.com/paper/1902.07987/) (2019)
- [Parsing Table Structures in the Wild](https://mlsanity.com/paper/2109.02199/) (2021)
- [Split, embed and merge: An accurate table structure recognizer](https://mlsanity.com/paper/2107.05214/) (2021)
- [PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML](https://mlsanity.com/paper/2105.01848/) (2021)
- [Neural Collaborative Graph Machines for Table Structure Recognition](https://mlsanity.com/paper/2111.13359/) (2021)

## Built on by (most cited first, 30 of 41)

- [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024)
- [OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition](https://mlsanity.com/paper/2403.19128/) (2024)
- [Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation](https://mlsanity.com/paper/2310.16809/) (2023)
- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models](https://mlsanity.com/paper/2502.16161/) (2025)
- [Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing](https://mlsanity.com/paper/2510.15349/) (2025)
- [GridFormer: Towards Accurate Table Structure Recognition via Grid Prediction](https://mlsanity.com/paper/2309.14962/) (2023)
- [UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining](https://mlsanity.com/paper/2403.04822/) (2024)
- [UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition](https://mlsanity.com/paper/2409.13148/) (2024)
- [Enhancing Table Recognition with Vision LLMs: A Benchmark and Neighbor-Guided Toolchain Reasoner](https://mlsanity.com/paper/2412.20662/) (2024)
- [TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism](https://mlsanity.com/paper/2501.11800/) (2025)
- [Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language Models](https://mlsanity.com/paper/2509.17589/) (2025)
- [SEMv3: A Fast and Robust Approach to Table Separation Line Detection](https://mlsanity.com/paper/2405.11862/) (2024)
- [Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training](https://mlsanity.com/paper/2603.23885/) (2026)
- [SPRINT: Script-agnostic Structure Recognition in Tables](https://mlsanity.com/paper/2503.11932/) (2025)
- [Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition](https://mlsanity.com/paper/2404.13268/) (2024)
- [DocFusion: A Unified Framework for Document Parsing Tasks](https://mlsanity.com/paper/2412.12505/) (2024)
- [TABLET: Table Structure Recognition using Encoder-only Transformers](https://mlsanity.com/paper/2506.07015/) (2025)
- [SCORE: A Semantic Evaluation Framework for Generative Document Parsing](https://mlsanity.com/paper/2509.19345/) (2025)
- [High-Performance Transformers for Table Structure Recognition Need Early Convolutions](https://mlsanity.com/paper/2311.05565/) (2023)
- [Vision Language Models for Spreadsheet Understanding: Challenges and Opportunities](https://mlsanity.com/paper/2405.16234/) (2024)
- [Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing](https://mlsanity.com/paper/2506.03197/) (2025)
- [Synthesizing Realistic Data for Table Recognition](https://mlsanity.com/paper/2404.11100/) (2024)
- [DREAM: Document Reconstruction via End-to-end Autoregressive Model](https://mlsanity.com/paper/2507.05805/) (2025)
- [SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia](https://mlsanity.com/paper/2603.15409/) (2026)
- [StrucTab: A Structured Optimization Framework for Table Parsing](https://mlsanity.com/paper/2606.29905/) (2026)
- [Rethinking Detection Based Table Structure Recognition for Visually Rich Document Images](https://mlsanity.com/paper/2312.00699/) (2023)
- [Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction in Financial Documents at Scale](https://mlsanity.com/paper/2507.08865/) (2025)
- [TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignment](https://mlsanity.com/paper/2603.22819/) (2026)
- [InstructTable: Improving Table Structure Recognition Through Instructions](https://mlsanity.com/paper/2604.02880/) (2026)

## Abstract (arXiv)

Table structure recognition aims to extract the logical and physical structure of unstructured table images into a machine-readable format. The latest end-to-end image-to-text approaches simultaneously predict the two structures by two decoders, where the prediction of the physical structure (the bounding boxes of the cells) is based on the representation of the logical structure. However, the previous methods struggle with imprecise bounding boxes as the logical representation lacks local visual information. To address this issue, we propose an end-to-end sequential modeling framework for table structure recognition called VAST. It contains a novel coordinate sequence decoder triggered by the representation of the non-empty cell from the logical structure decoder. In the coordinate sequence decoder, we model the bounding box coordinates as a language sequence, where the left, top, right and bottom coordinates are decoded sequentially to leverage the inter-coordinate dependency. Furthermore, we propose an auxiliary visual-alignment loss to enforce the logical representation of the non-empty cells to contain more local visual details, which helps produce better cell bounding boxes. Extensive experiments demonstrate that our proposed method can achieve state-of-the-art results in both logical and physical structure recognition. The ablation study also validates that the proposed coordinate sequence decoder and the visual-alignment loss are the keys to the success of our method.