# Table Search Using a Deep Contextualized Language Model

arXiv: https://arxiv.org/abs/2005.09207 · HTML: https://mlsanity.com/paper/2005.09207/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2020.

## Built on by (most cited first, 22 of 22)

- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training](https://mlsanity.com/paper/2211.02816/) (2022)
- [Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval](https://mlsanity.com/paper/2404.09889/) (2024)
- [Retrieving Complex Tables with Multi-Granular Graph Representation Learning](https://mlsanity.com/paper/2105.01736/) (2021)
- [Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA](https://mlsanity.com/paper/2210.05197/) (2022)
- [Neural ranking models for document retrieval](https://mlsanity.com/paper/2102.11903/) (2021)
- [DeepJoin: Joinable Table Discovery with Pre-trained Language Models](https://mlsanity.com/paper/2212.07588/) (2022)
- [Multi-modal Retrieval of Tables and Texts Using Tri-encoder Models](https://mlsanity.com/paper/2108.04049/) (2021)
- [Table Retrieval May Not Necessitate Table-specific Model Design](https://mlsanity.com/paper/2205.09843/) (2022)
- [Time Series Language Model for Descriptive Caption Generation](https://mlsanity.com/paper/2501.01832/) (2025)
- [DAME: Domain Adaptation for Matching Entities](https://mlsanity.com/paper/2204.09244/) (2022)
- [Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research](https://mlsanity.com/paper/2110.08975/) (2021)
- [Multi-Field Adaptive Retrieval](https://mlsanity.com/paper/2410.20056/) (2024)
- [Empowering Tabular Data Preparation with Language Models: Why and How?](https://mlsanity.com/paper/2508.01556/) (2025)
- [Absformer: Transformer-based Model for Unsupervised Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2306.04787/) (2023)
- [Enhancing Open-Domain Table Question Answering via Syntax- and Structure-aware Dense Retrieval](https://mlsanity.com/paper/2309.10506/) (2023)
- [BERTVision -- A Parameter-Efficient Approach for Question Answering](https://mlsanity.com/paper/2202.12210/) (2022)
- [Contextualized Token Discrimination for Speech Search Query Correction](https://mlsanity.com/paper/2509.04393/) (2025)
- [CORE-T: COherent REtrieval of Tables for Text-to-SQL](https://mlsanity.com/paper/2601.13111/) (2026)
- [FollowTable: A Benchmark for Instruction-Following Table Retrieval](https://mlsanity.com/paper/2605.00400/) (2026)
- [PIPER: Content-Based Table Search via profiling and LLM-Generated Pseudoqueries](https://mlsanity.com/paper/2605.18199/) (2026)
- [Polaris: Learning to Generate Table Descriptions from Retrieval Feedback](https://mlsanity.com/paper/2608.17171/) (2026)

## Abstract (arXiv)

Pretrained contextualized language models such as BERT have achieved impressive results on various natural language processing benchmarks. Benefiting from multiple pretraining tasks and large scale training corpora, pretrained models can capture complex syntactic word relations. In this paper, we use the deep contextualized language model BERT for the task of ad hoc table retrieval. We investigate how to encode table content considering the table structure and input length limit of BERT. We also propose an approach that incorporates features from prior literature on table retrieval and jointly trains them with BERT. In experiments on public datasets, we show that our best approach can outperform the previous state-of-the-art method and BERT baselines with a large margin under different evaluation metrics.