# Split, embed and merge: An accurate table structure recognizer

arXiv: https://arxiv.org/abs/2107.05214 · HTML: https://mlsanity.com/paper/2107.05214/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 62 references that are in the map.
- First posted on arXiv 12 Jul 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes](https://mlsanity.com/paper/1908.08207/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes](https://mlsanity.com/paper/1807.01544/) (2018)
- [Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting](https://mlsanity.com/paper/2007.09482/) (2020)
- [Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA](https://mlsanity.com/paper/1911.06258/) (2019)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [Complicated Table Structure Recognition](https://mlsanity.com/paper/1908.04729/) (2019)
- [Table Structure Recognition using Top-Down and Bottom-Up Cues](https://mlsanity.com/paper/2010.04565/) (2020)
- [TextMountain: Accurate Scene Text Detection via Instance Segmentation](https://mlsanity.com/paper/1811.12786/) (2018)
- [Accuracy vs. Complexity: A Trade-off in Visual Question Answering Models](https://mlsanity.com/paper/2001.07059/) (2020)

## Built on by (most cited first, 30 of 42)

- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modeling](https://mlsanity.com/paper/2303.06949/) (2023)
- [Optimized Table Tokenization for Table Structure Recognition](https://mlsanity.com/paper/2305.03393/) (2023)
- [Robust Table Detection and Structure Recognition from Heterogeneous Document Images](https://mlsanity.com/paper/2203.09056/) (2022)
- [TSRFormer: Table Structure Recognition with Transformers](https://mlsanity.com/paper/2208.04921/) (2022)
- [Multimodal Pre-training Based on Graph Attention Network for Document Understanding](https://mlsanity.com/paper/2203.13530/) (2022)
- [TRUST: An Accurate and End-to-End Table structure Recognizer Using Splitting-based Transformers](https://mlsanity.com/paper/2208.14687/) (2022)
- [LORE: Logical Location Regression Network for Table Structure Recognition](https://mlsanity.com/paper/2303.03730/) (2023)
- [Visual Understanding of Complex Table Structures from Document Images](https://mlsanity.com/paper/2111.07129/) (2021)
- [Deep learning for table detection and structure recognition: A survey](https://mlsanity.com/paper/2211.08469/) (2022)
- [An End-to-End Multi-Task Learning Model for Image-based Table Recognition](https://mlsanity.com/paper/2303.08648/) (2023)
- [GridFormer: Towards Accurate Table Structure Recognition via Grid Prediction](https://mlsanity.com/paper/2309.14962/) (2023)
- [SEMv2: Table Separation Line Detection Based on Instance Segmentation](https://mlsanity.com/paper/2303.04384/) (2023)
- [Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer](https://mlsanity.com/paper/2303.11615/) (2023)
- [UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition](https://mlsanity.com/paper/2409.13148/) (2024)
- [Rethinking Image-based Table Recognition Using Weakly Supervised Methods](https://mlsanity.com/paper/2303.07641/) (2023)
- [TFLOP: Table Structure Recognition Framework with Layout Pointer Mechanism](https://mlsanity.com/paper/2501.11800/) (2025)
- [Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language Models](https://mlsanity.com/paper/2509.17589/) (2025)
- [Multimodal Tree Decoder for Table of Contents Extraction in Document Images](https://mlsanity.com/paper/2212.02896/) (2022)
- [LORE++: Logical Location Regression Network for Table Structure Recognition with Pre-training](https://mlsanity.com/paper/2401.01522/) (2024)
- [SEMv3: A Fast and Robust Approach to Table Separation Line Detection](https://mlsanity.com/paper/2405.11862/) (2024)
- [TRACE: Table Reconstruction Aligned to Corner and Edges](https://mlsanity.com/paper/2305.00630/) (2023)
- [SPRINT: Script-agnostic Structure Recognition in Tables](https://mlsanity.com/paper/2503.11932/) (2025)
- [Multi-Cell Decoder and Mutual Learning for Table Structure and Character Recognition](https://mlsanity.com/paper/2404.13268/) (2024)
- [TABLET: Table Structure Recognition using Encoder-only Transformers](https://mlsanity.com/paper/2506.07015/) (2025)
- [StrucTab: A Structured Optimization Framework for Table Parsing](https://mlsanity.com/paper/2606.29905/) (2026)
- [Rethinking Detection Based Table Structure Recognition for Visually Rich Document Images](https://mlsanity.com/paper/2312.00699/) (2023)
- [Bidirectional Trained Tree-Structured Decoder for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2401.00435/) (2023)
- [PdfTable: A Unified Toolkit for Deep Learning-Based Table Extraction](https://mlsanity.com/paper/2409.05125/) (2024)
- [TableCenterNet: A one-stage network for table structure recognition](https://mlsanity.com/paper/2504.17522/) (2025)

## Abstract (arXiv)

Table structure recognition is an essential part for making machines understand tables. Its main task is to recognize the internal structure of a table. However, due to the complexity and diversity in their structure and style, it is very difficult to parse the tabular data into the structured format which machines can understand easily, especially for complex tables. In this paper, we introduce Split, Embed and Merge (SEM), an accurate table structure recognizer. Our model takes table images as input and can correctly recognize the structure of tables, whether they are simple or a complex tables. SEM is mainly composed of three parts, splitter, embedder and merger. In the first stage, we apply the splitter to predict the potential regions of the table row (column) separators, and obtain the fine grid structure of the table. In the second stage, by taking a full consideration of the textual information in the table, we fuse the output features for each table grid from both vision and language modalities. Moreover, we achieve a higher precision in our experiments through adding additional semantic features. Finally, we process the merging of these basic table grids in a self-regression manner. The correspondent merging results is learned through the attention mechanism. In our experiments, SEM achieves an average F1-Measure of 97.11% on the SciTSR dataset which outperforms other methods by a large margin. We also won the first place in the complex table and third place in all tables in ICDAR 2021 Competition on Scientific Literature Parsing, Task-B. Extensive experiments on other publicly available datasets demonstrate that our model achieves state-of-the-art.