# BERT Meets Relational DB: Contextual Representations of Relational Databases

arXiv: https://arxiv.org/abs/2104.14914 · HTML: https://mlsanity.com/paper/2104.14914/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval](https://mlsanity.com/paper/1906.00041/) (2019)
- [TAPER: Time-Aware Patient EHR Representation](https://mlsanity.com/paper/1908.03971/) (2019)
- [Ad Hoc Table Retrieval using Semantic Similarity](https://mlsanity.com/paper/1802.06159/) (2018)
- [TURL: Table Understanding through Representation Learning](https://mlsanity.com/paper/2006.14806/) (2020)
- [Unlocking New York City Crime Insights using Relational Database Embeddings](https://mlsanity.com/paper/2005.09617/) (2020)
- [Recommending Related Tables](https://mlsanity.com/paper/1907.03595/) (2019)

## Built on by (most cited first, 1 of 1)

- [Learning Neural Models for Continuous-Time Sequences](https://mlsanity.com/paper/2111.07189/) (2021)

## Abstract (arXiv)

In this paper, we address the problem of learning low dimension representation of entities on relational databases consisting of multiple tables. Embeddings help to capture semantics encoded in the database and can be used in a variety of settings like auto-completion of tables, fully-neural query processing of relational joins queries, seamlessly handling missing values, and more. Current work is restricted to working with just single table, or using pretrained embeddings over an external corpus making them unsuitable for use in real-world databases. In this work, we look into ways of using these attention-based model to learn embeddings for entities in the relational database. We are inspired by BERT style pretraining methods and are interested in observing how they can be extended for representation learning on structured databases. We evaluate our approach of the autocompletion of relational databases and achieve improvement over standard baselines.