# UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition

arXiv: https://arxiv.org/abs/2409.13148 · HTML: https://mlsanity.com/paper/2409.13148/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [Pix2seq: A Language Modeling Framework for Object Detection](https://mlsanity.com/paper/2109.10852/) (2021)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [PubTables-1M: Towards comprehensive table extraction from unstructured documents](https://mlsanity.com/paper/2110.00061/) (2021)
- [TableFormer: Table Structure Understanding with Transformers](https://mlsanity.com/paper/2203.01017/) (2022)
- [Complicated Table Structure Recognition](https://mlsanity.com/paper/1908.04729/) (2019)
- [ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding](https://mlsanity.com/paper/2210.06155/) (2022)
- [Table Structure Recognition using Top-Down and Bottom-Up Cues](https://mlsanity.com/paper/2010.04565/) (2020)
- [LGPMA: Complicated Table Structure Recognition with Local and Global Pyramid Mask Alignment](https://mlsanity.com/paper/2105.06224/) (2021)
- [Parsing Table Structures in the Wild](https://mlsanity.com/paper/2109.02199/) (2021)
- [Split, embed and merge: An accurate table structure recognizer](https://mlsanity.com/paper/2107.05214/) (2021)
- [Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modeling](https://mlsanity.com/paper/2303.06949/) (2023)
- [Neural Collaborative Graph Machines for Table Structure Recognition](https://mlsanity.com/paper/2111.13359/) (2021)
- [Robust Table Detection and Structure Recognition from Heterogeneous Document Images](https://mlsanity.com/paper/2203.09056/) (2022)
- [GriTS: Grid table similarity metric for table structure recognition](https://mlsanity.com/paper/2203.12555/) (2022)
- [TRUST: An Accurate and End-to-End Table structure Recognizer Using Splitting-based Transformers](https://mlsanity.com/paper/2208.14687/) (2022)
- [LORE: Logical Location Regression Network for Table Structure Recognition](https://mlsanity.com/paper/2303.03730/) (2023)
- [SEMv2: Table Separation Line Detection Based on Instance Segmentation](https://mlsanity.com/paper/2303.04384/) (2023)
- [Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer](https://mlsanity.com/paper/2303.11615/) (2023)
- [SEMv3: A Fast and Robust Approach to Table Separation Line Detection](https://mlsanity.com/paper/2405.11862/) (2024)

## Built on by (most cited first, 11 of 11)

- [TABLET: Table Structure Recognition using Encoder-only Transformers](https://mlsanity.com/paper/2506.07015/) (2025)
- [TableCenterNet: A one-stage network for table structure recognition](https://mlsanity.com/paper/2504.17522/) (2025)
- [TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignment](https://mlsanity.com/paper/2603.22819/) (2026)
- [Hierarchical structure understanding in complex tables with VLLMs: a benchmark and experiments](https://mlsanity.com/paper/2511.08298/) (2025)
- [How to Utilize Complementary Vision-Text Information for 2D Structure Understanding](https://mlsanity.com/paper/2603.16245/) (2026)
- [TableSeq: Unified Generation of Structure, Content, and Layout](https://mlsanity.com/paper/2604.16070/) (2026)
- [FastTab: A Fast Table Recognizer with a Tiny Recursive Module and 1D Transformers](https://mlsanity.com/paper/2605.22422/) (2026)
- [ConRTF: Edge-Constrained Boundary Distribution Refinement for Realtime TransFormer Table Structure Recognition](https://mlsanity.com/paper/2607.00734/) (2026)
- [Active Learning for Cascaded Object Detection: Balancing Coverage and Uncertainty in Table Extraction Pipelines](https://mlsanity.com/paper/2607.00747/) (2026)
- [From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsing](https://mlsanity.com/paper/2608.09842/) (2026)
- [Learning to Ground Before Reading: Unified PCB Engineering Drawing Parsing with Compact Vision-Language Models](https://mlsanity.com/paper/2608.29268/) (2026)

## Abstract (arXiv)

In the digital era, table structure recognition technology is a critical tool for processing and analyzing large volumes of tabular data. Previous methods primarily focus on visual aspects of table structure recovery but often fail to effectively comprehend the textual semantics within tables, particularly for descriptive textual cells. In this paper, we introduce UniTabNet, a novel framework for table structure parsing based on the image-to-text model. UniTabNet employs a ``divide-and-conquer'' strategy, utilizing an image-to-text model to decouple table cells and integrating both physical and logical decoders to reconstruct the complete table structure. We further enhance our framework with the Vision Guider, which directs the model's focus towards pertinent areas, thereby boosting prediction accuracy. Additionally, we introduce the Language Guider to refine the model's capability to understand textual semantics in table images. Evaluated on prominent table structure datasets such as PubTabNet, PubTables1M, WTW, and iFLYTAB, UniTabNet achieves a new state-of-the-art performance, demonstrating the efficacy of our approach. The code will also be made publicly available.