# Can Deep Neural Networks Predict Data Correlations from Column Names?

arXiv: https://arxiv.org/abs/2107.04553 · HTML: https://mlsanity.com/paper/2107.04553/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 74 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning](https://mlsanity.com/paper/1711.04436/) (2017)
- [A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization](https://mlsanity.com/paper/1902.01069/) (2019)
- [Neo: A Learned Query Optimizer](https://mlsanity.com/paper/1904.03711/) (2019)
- [Learned Cardinalities: Estimating Correlated Joins with Deep Learning](https://mlsanity.com/paper/1809.00677/) (2018)
- [An End-to-End Learning-based Cost Estimator](https://mlsanity.com/paper/1906.02560/) (2019)
- [Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching](https://mlsanity.com/paper/2004.12297/) (2020)
- [RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparation](https://mlsanity.com/paper/2012.02469/) (2020)
- [SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning](https://mlsanity.com/paper/1901.05152/) (2019)
- [QuickSel: Quick Selectivity Learning with Mixture Models](https://mlsanity.com/paper/1812.10568/) (2018)
- [UDO: Universal Database Optimization using Reinforcement Learning](https://mlsanity.com/paper/2104.01744/) (2021)
- [DataPrep.EDA: Task-Centric Exploratory Data Analysis for Statistical Modeling in Python](https://mlsanity.com/paper/2104.00841/) (2021)

## Built on by (most cited first, 2 of 2)

- [CHORUS: Foundation Models for Unified Data Discovery and Exploration](https://mlsanity.com/paper/2306.09610/) (2023)
- [METAM: Goal-Oriented Data Discovery](https://mlsanity.com/paper/2304.09068/) (2023)

## Abstract (arXiv)

Recent publications suggest using natural language analysis on database schema elements to guide tuning and profiling efforts. The underlying hypothesis is that state-of-the-art language processing methods, so-called language models, are able to extract information on data properties from schema text. This paper examines that hypothesis in the context of data correlation analysis: is it possible to find column pairs with correlated data by analyzing their names via language models? First, the paper introduces a novel benchmark for data correlation analysis, created by analyzing thousands of Kaggle data sets (and available for download). Second, it uses that data to study the ability of language models to predict correlation, based on column names. The analysis covers different language models, various correlation metrics, and a multitude of accuracy metrics. It pinpoints factors that contribute to successful predictions, such as the length of column names as well as the ratio of words. Finally, revthe study analyzes the impact of column types on prediction performance. The results show that schema text can be a useful source of information and inform future research efforts, targeted at NLP-enhanced database tuning and data profiling.