# TURL: Table Understanding through Representation Learning

arXiv: https://arxiv.org/abs/2006.14806 · HTML: https://mlsanity.com/paper/2006.14806/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2020.

## Built on by (most cited first, 14 of 14)

- [Benchmarking Multimodal AutoML for Tabular Data with Text Fields](https://mlsanity.com/paper/2111.02705/) (2021)
- [Multi-modal Retrieval of Tables and Texts Using Tri-encoder Models](https://mlsanity.com/paper/2108.04049/) (2021)
- [Unsupervised Matching of Data and Text](https://mlsanity.com/paper/2112.08776/) (2021)
- [TableMoE: Neuro-Symbolic Routing for Structured Expert Reasoning in Multimodal Table Understanding](https://mlsanity.com/paper/2506.21393/) (2025)
- [A Numerical Reasoning Question Answering System with Fine-grained Retriever and the Ensemble of Multiple Generators for FinQA](https://mlsanity.com/paper/2206.08506/) (2022)
- [Entity Linking in Tabular Data Needs the Right Attention](https://mlsanity.com/paper/2207.01937/) (2022)
- [Embeddings for Tabular Data: A Survey](https://mlsanity.com/paper/2302.11777/) (2023)
- [BERT Meets Relational DB: Contextual Representations of Relational Databases](https://mlsanity.com/paper/2104.14914/) (2021)
- [Making Table Understanding Work in Practice](https://mlsanity.com/paper/2109.05173/) (2021)
- [Rows from Many Sources: Enriching row completions from Wikidata with a pre-trained Language Model](https://mlsanity.com/paper/2204.07014/) (2022)
- [Relation Extraction from Tables using Artificially Generated Metadata](https://mlsanity.com/paper/2108.10750/) (2021)
- [Russian Web Tables: A Public Corpus of Web Tables for Russian Language Based on Wikipedia](https://mlsanity.com/paper/2210.06353/) (2022)
- [Active Learning with Tabular Language Models](https://mlsanity.com/paper/2211.04128/) (2022)
- [Progressive Feature Upgrade in Semi-supervised Learning on Tabular Domain](https://mlsanity.com/paper/2212.00892/) (2022)

## Abstract (arXiv)

Relational tables on the Web store a vast amount of knowledge. Owing to the wealth of such tables, there has been tremendous progress on a variety of tasks in the area of table understanding. However, existing work generally relies on heavily-engineered task-specific features and model architectures. In this paper, we present TURL, a novel framework that introduces the pre-training/fine-tuning paradigm to relational Web tables. During pre-training, our framework learns deep contextualized representations on relational tables in an unsupervised manner. Its universal model design with pre-trained representations can be applied to a wide range of tasks with minimal task-specific fine-tuning. Specifically, we propose a structure-aware Transformer encoder to model the row-column structure of relational tables, and present a new Masked Entity Recovery (MER) objective for pre-training to capture the semantics and knowledge in large-scale unlabeled data. We systematically evaluate TURL with a benchmark consisting of 6 different tasks for table understanding (e.g., relation extraction, cell filling). We show that TURL generalizes well to all tasks and substantially outperforms existing methods in almost all instances.