# GitTables: A Large-Scale Corpus of Relational Tables

arXiv: https://arxiv.org/abs/2106.07258 · HTML: https://mlsanity.com/paper/2106.07258/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 71 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models](https://mlsanity.com/paper/2102.02503/) (2021)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Sherlock: A Deep Learning Approach to Semantic Data Type Detection](https://mlsanity.com/paper/1905.10688/) (2019)
- [Database Meets Deep Learning: Challenges and Opportunities](https://mlsanity.com/paper/1906.08986/) (2019)
- [VizNet: Towards A Large-Scale Visualization Learning and Benchmarking Repository](https://mlsanity.com/paper/1905.04616/) (2019)
- [TCN: Table Convolutional Network for Web Table Interpretation](https://mlsanity.com/paper/2102.09460/) (2021)
- [Wrangling Messy CSV Files by Detecting Row and Type Patterns](https://mlsanity.com/paper/1811.11242/) (2018)
- [Sato: Contextual Semantic Type Detection in Tables](https://mlsanity.com/paper/1911.06311/) (2019)

## Built on by (most cited first, 30 of 45)

- [Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study](https://mlsanity.com/paper/2305.13062/) (2023)
- [OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale](https://mlsanity.com/paper/2503.02240/) (2025)
- [TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios](https://mlsanity.com/paper/2403.19318/) (2024)
- [CARTE: Pretraining and Transfer for Tabular Learning](https://mlsanity.com/paper/2402.16785/) (2024)
- [Large Language Model for Table Processing: A Survey](https://mlsanity.com/paper/2402.05121/) (2024)
- [Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Tasks](https://mlsanity.com/paper/2201.09745/) (2022)
- [Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Data](https://mlsanity.com/paper/2507.03971/) (2025)
- [Importance of Synthesizing High-quality Data for Text-to-SQL Parsing](https://mlsanity.com/paper/2212.08785/) (2022)
- [CHORUS: Foundation Models for Unified Data Discovery and Exploration](https://mlsanity.com/paper/2306.09610/) (2023)
- [TabLib: A Dataset of 627M Tables with Context](https://mlsanity.com/paper/2310.07875/) (2023)
- [Towards Cross-Table Masked Pretraining for Web Data Mining](https://mlsanity.com/paper/2307.04308/) (2023)
- [TARGET: Benchmarking Table Retrieval for Generative Tasks](https://mlsanity.com/paper/2505.11545/) (2025)
- [ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models](https://mlsanity.com/paper/2310.18208/) (2023)
- [Tackling prediction tasks in relational databases with LLMs](https://mlsanity.com/paper/2411.11829/) (2024)
- [NameGuess: Column Name Expansion for Tabular Data](https://mlsanity.com/paper/2310.13196/) (2023)
- [DBCopilot: Natural Language Querying over Massive Databases via Schema Routing](https://mlsanity.com/paper/2312.03463/) (2023)
- [Granite Embedding R2 Models](https://mlsanity.com/paper/2508.21085/) (2025)
- [Towards Foundation Models for Relational Databases [Vision Paper]](https://mlsanity.com/paper/2305.15321/) (2023)
- [Mind the Data Gap: Bridging LLMs to Enterprise Data Integration](https://mlsanity.com/paper/2412.20331/) (2024)
- [PMLBmini: A Tabular Classification Benchmark Suite for Data-Scarce Applications](https://mlsanity.com/paper/2409.01635/) (2024)
- [Position: Foundation Models for Tabular Data within Systemic Contexts Need Grounding](https://mlsanity.com/paper/2505.19825/) (2025)
- [Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks](https://mlsanity.com/paper/2604.21696/) (2026)
- [Retrieve, Merge, Predict: Augmenting Tables with Data Lakes](https://mlsanity.com/paper/2402.06282/) (2024)
- [AdaTyper: Adaptive Semantic Column Type Detection](https://mlsanity.com/paper/2311.13806/) (2023)
- [TabularFM: An Open Framework For Tabular Foundational Models](https://mlsanity.com/paper/2406.09837/) (2024)
- [Making Table Understanding Work in Practice](https://mlsanity.com/paper/2109.05173/) (2021)
- [Towards Multi-Modal DBMSs for Seamless Querying of Texts and Tables](https://mlsanity.com/paper/2304.13559/) (2023)
- [Generative Benchmark Creation for Table Union Search](https://mlsanity.com/paper/2308.03883/) (2023)
- [Towards Universal Dense Blocking for Entity Resolution](https://mlsanity.com/paper/2404.14831/) (2024)
- [TabSketchFM: Sketch-based Tabular Representation Learning for Data Discovery over Data Lakes](https://mlsanity.com/paper/2407.01619/) (2024)

## Abstract (arXiv)

The success of deep learning has sparked interest in improving relational table tasks, like data preparation and search, with table representation models trained on large table corpora. Existing table corpora primarily contain tables extracted from HTML pages, limiting the capability to represent offline database tables. To train and evaluate high-capacity models for applications beyond the Web, we need resources with tables that resemble relational database tables. Here we introduce GitTables, a corpus of 1M relational tables extracted from GitHub. Our continuing curation aims at growing the corpus to at least 10M tables. Analyses of GitTables show that its structure, content, and topical coverage differ significantly from existing table corpora. We annotate table columns in GitTables with semantic types, hierarchical relations and descriptions from Schema.org and DBpedia. The evaluation of our annotation pipeline on the T2Dv2 benchmark illustrates that our approach provides results on par with human annotations. We present three applications of GitTables, demonstrating its value for learned semantic type detection models, schema completion methods, and benchmarks for table-to-KG matching, data search, and preparation. We make the corpus and code available at https://gittables.github.io.