# Sherlock: A Deep Learning Approach to Semantic Data Type Detection

arXiv: https://arxiv.org/abs/1905.10688 · HTML: https://mlsanity.com/paper/1905.10688/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2019.

## Built on by (most cited first, 30 of 74)

- [Fine-tuning language models to find agreement among humans with diverse preferences](https://mlsanity.com/paper/2211.15006/) (2022)
- [TableLlama: Towards Open Large Generalist Models for Tables](https://mlsanity.com/paper/2311.09206/) (2023)
- [TABBIE: Pretrained Representations of Tabular Data](https://mlsanity.com/paper/2105.02584/) (2021)
- [CARTE: Pretraining and Transfer for Tabular Learning](https://mlsanity.com/paper/2402.16785/) (2024)
- [Table-GPT: Table-tuned GPT for Diverse Table Tasks](https://mlsanity.com/paper/2310.09263/) (2023)
- [GitTables: A Large-Scale Corpus of Relational Tables](https://mlsanity.com/paper/2106.07258/) (2021)
- [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021)
- [HYTREL: Hypergraph-enhanced Tabular Data Representation Learning](https://mlsanity.com/paper/2307.08623/) (2023)
- [LightAutoML: AutoML Solution for a Large Financial Services Ecosystem](https://mlsanity.com/paper/2109.01528/) (2021)
- [TCN: Table Convolutional Network for Web Table Interpretation](https://mlsanity.com/paper/2102.09460/) (2021)
- [CHORUS: Foundation Models for Unified Data Discovery and Exploration](https://mlsanity.com/paper/2306.09610/) (2023)
- [TabLib: A Dataset of 627M Tables with Context](https://mlsanity.com/paper/2310.07875/) (2023)
- [Towards Cross-Table Masked Pretraining for Web Data Mining](https://mlsanity.com/paper/2307.04308/) (2023)
- [Column Type Annotation using ChatGPT](https://mlsanity.com/paper/2306.00745/) (2023)
- [SpreadsheetLLM: Encoding Spreadsheets for Large Language Models](https://mlsanity.com/paper/2407.09025/) (2024)
- [TARGET: Benchmarking Table Retrieval for Generative Tasks](https://mlsanity.com/paper/2505.11545/) (2025)
- [DeepJoin: Joinable Table Discovery with Pre-trained Language Models](https://mlsanity.com/paper/2212.07588/) (2022)
- [ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models](https://mlsanity.com/paper/2310.18208/) (2023)
- [Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AI](https://mlsanity.com/paper/2407.21523/) (2024)
- [MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2506.05587/) (2025)
- [NameGuess: Column Name Expansion for Tabular Data](https://mlsanity.com/paper/2310.13196/) (2023)
- [Observatory: Characterizing Embeddings of Relational Tables](https://mlsanity.com/paper/2310.07736/) (2023)
- [LakeBench: Benchmarks for Data Discovery over Data Lakes](https://mlsanity.com/paper/2307.04217/) (2023)
- [Knowledge Engineering using Large Language Models](https://mlsanity.com/paper/2310.00637/) (2023)
- [Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs](https://mlsanity.com/paper/2601.17058/) (2026)
- [AnaMeta: A Table Understanding Dataset of Field Metadata Knowledge Shared by Multi-dimensional Data Analysis Tasks](https://mlsanity.com/paper/2209.00946/) (2022)
- [Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes](https://mlsanity.com/paper/2104.04659/) (2021)
- [AutoG: Towards automatic graph construction from tabular data](https://mlsanity.com/paper/2501.15282/) (2025)
- [DataVinci: Learning Syntactic and Semantic String Repairs](https://mlsanity.com/paper/2308.10922/) (2023)
- [KAER: A Knowledge Augmented Pre-Trained Language Model for Entity Resolution](https://mlsanity.com/paper/2301.04770/) (2023)

## Abstract (arXiv)

Correctly detecting the semantic type of data columns is crucial for data science tasks such as automated data cleaning, schema matching, and data discovery. Existing data preparation and analysis systems rely on dictionary lookups and regular expression matching to detect semantic types. However, these matching-based approaches often are not robust to dirty data and only detect a limited number of types. We introduce Sherlock, a multi-input deep neural network for detecting semantic types. We train Sherlock on 686,765 data columns retrieved from the VizNet corpus by matching 78 semantic types from DBpedia to column headers. We characterize each matched column with 1,588 features describing the statistical properties, character distributions, word embeddings, and paragraph vectors of column values. Sherlock achieves a support-weighted F1 score of 0.89, exceeding that of machine learning baselines, dictionary and regular expression benchmarks, and the consensus of crowdsourced annotations.