# TabularNet: A Neural Network Architecture for Understanding Semantic Structures of Tabular Data

arXiv: https://arxiv.org/abs/2106.03096 · HTML: https://mlsanity.com/paper/2106.03096/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [A Convolutional Neural Network for Modelling Sentences](https://mlsanity.com/paper/1404.2188/) (2014)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [DANE: Domain Adaptive Network Embedding](https://mlsanity.com/paper/1906.00684/) (2019)
- [Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval](https://mlsanity.com/paper/1906.00041/) (2019)
- [ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks](https://mlsanity.com/paper/1505.00393/) (2015)
- [Semantic Classification of Tabular Datasets via Character-Level Convolutional Neural Networks](https://mlsanity.com/paper/1901.08456/) (2019)

## Built on by (most cited first, 30 of 36)

- [GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily](https://mlsanity.com/paper/2110.15777/) (2021)
- [Griffin: Towards a Graph-Centric Relational Database Foundation Model](https://mlsanity.com/paper/2505.05568/) (2025)
- [Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Tasks](https://mlsanity.com/paper/2201.09745/) (2022)
- [PTaRL: Prototype-based Tabular Representation Learning via Space Calibration](https://mlsanity.com/paper/2407.05364/) (2024)
- [Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach](https://mlsanity.com/paper/2209.08264/) (2022)
- [Hopular: Modern Hopfield Networks for Tabular Data](https://mlsanity.com/paper/2206.00664/) (2022)
- [Text-to-Image Generation for Abstract Concepts](https://mlsanity.com/paper/2309.14623/) (2023)
- [A Survey on Deep Tabular Learning](https://mlsanity.com/paper/2410.12034/) (2024)
- [GFS: Graph-based Feature Synthesis for Prediction over Relational Databases](https://mlsanity.com/paper/2312.02037/) (2023)
- [Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples](https://mlsanity.com/paper/2307.14565/) (2023)
- [Relational In-Context Learning via Synthetic Pre-training with Structural Prior](https://mlsanity.com/paper/2603.03805/) (2026)
- [Graph Neural Networks for Tabular Data Learning: A Survey with Taxonomy and Directions](https://mlsanity.com/paper/2401.02143/) (2024)
- [MM-GNN: Mix-Moment Graph Neural Network towards Modeling Neighborhood Feature Distribution](https://mlsanity.com/paper/2208.07012/) (2022)
- [Predicting the Silent Majority on Graphs: Knowledge Transferable Graph Neural Network](https://mlsanity.com/paper/2302.00873/) (2023)
- [Why LLMs Are Bad at Synthetic Table Generation (and what to do about it)](https://mlsanity.com/paper/2406.14541/) (2024)
- [TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning](https://mlsanity.com/paper/2409.11724/) (2024)
- [Neuron Campaign for Initialization Guided by Information Bottleneck Theory](https://mlsanity.com/paper/2108.06530/) (2021)
- [Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification](https://mlsanity.com/paper/2205.15008/) (2022)
- [Interpretable Graph Neural Networks for Tabular Data](https://mlsanity.com/paper/2308.08945/) (2023)
- [TableTalk: Scaffolding Spreadsheet Development with a Language Agent](https://mlsanity.com/paper/2502.09787/) (2025)
- [Neuron with Steady Response Leads to Better Generalization](https://mlsanity.com/paper/2111.15414/) (2021)
- [Tree-Regularized Tabular Embeddings](https://mlsanity.com/paper/2403.00963/) (2024)
- [Embeddings for Tabular Data: A Survey](https://mlsanity.com/paper/2302.11777/) (2023)
- [Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima](https://mlsanity.com/paper/2208.11873/) (2022)
- [AdaMedGraph: Adaboosting Graph Neural Networks for Personalized Medicine](https://mlsanity.com/paper/2311.14304/) (2023)
- [TAROT: A Hierarchical Framework with Multitask Co-Pretraining on Semi-Structured Data towards Effective Person-Job Fit](https://mlsanity.com/paper/2401.07525/) (2024)
- [Beyond Text-to-SQL for IoT Defense: A Comprehensive Framework for Querying and Classifying IoT Threats](https://mlsanity.com/paper/2406.17574/) (2024)
- [Interpretable Graph Neural Networks for Heterogeneous Tabular Data](https://mlsanity.com/paper/2408.07661/) (2024)
- [LLM Agents for Interactive Exploration of Historical Cadastre Data: Framework and Application to Venice](https://mlsanity.com/paper/2505.17148/) (2025)
- [Probabilistic Hash Embeddings for Online Learning of Categorical Features](https://mlsanity.com/paper/2511.20893/) (2025)

## Abstract (arXiv)

Tabular data are ubiquitous for the widespread applications of tables and hence have attracted the attention of researchers to extract underlying information. One of the critical problems in mining tabular data is how to understand their inherent semantic structures automatically. Existing studies typically adopt Convolutional Neural Network (CNN) to model the spatial information of tabular structures yet ignore more diverse relational information between cells, such as the hierarchical and paratactic relationships. To simultaneously extract spatial and relational information from tables, we propose a novel neural network architecture, TabularNet. The spatial encoder of TabularNet utilizes the row/column-level Pooling and the Bidirectional Gated Recurrent Unit (Bi-GRU) to capture statistical information and local positional correlation, respectively. For relational information, we design a new graph construction method based on the WordNet tree and adopt a Graph Convolutional Network (GCN) based encoder that focuses on the hierarchical and paratactic relationships between cells. Our neural network architecture can be a unified neural backbone for different understanding tasks and utilized in a multitask scenario. We conduct extensive experiments on three classification tasks with two real-world spreadsheet data sets, and the results demonstrate the effectiveness of our proposed TabularNet over state-of-the-art baselines.