# Semantic Labeling Using a Deep Contextualized Language Model

arXiv: https://arxiv.org/abs/2010.16037 · HTML: https://mlsanity.com/paper/2010.16037/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021)
- [Neural ranking models for document retrieval](https://mlsanity.com/paper/2102.11903/) (2021)
- [DAME: Domain Adaptation for Matching Entities](https://mlsanity.com/paper/2204.09244/) (2022)
- [Enhancing Table Recognition with Vision LLMs: A Benchmark and Neighbor-Guided Toolchain Reasoner](https://mlsanity.com/paper/2412.20662/) (2024)
- [Absformer: Transformer-based Model for Unsupervised Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2306.04787/) (2023)
- [Fine-Tuning BERT for Automatic ADME Semantic Labeling in FDA Drug Labeling to Enhance Product-Specific Guidance Assessment](https://mlsanity.com/paper/2207.12376/) (2022)
- [TAGIFY: LLM-powered Tagging Interface for Improved Data Findability on OGD portals](https://mlsanity.com/paper/2407.18764/) (2024)
- [TeleDoCTR: Domain-Specific and Contextual Troubleshooting for Telecommunications](https://mlsanity.com/paper/2601.00691/) (2026)
- [A Survey on Semantics in Automated Data Science](https://mlsanity.com/paper/2205.08018/) (2022)
- [A Vision for Semantically Enriched Data Science](https://mlsanity.com/paper/2303.01378/) (2023)
- [ZTab: Domain-based Zero-shot Annotation for Table Columns](https://mlsanity.com/paper/2603.11436/) (2026)

## Abstract (arXiv)

Generating schema labels automatically for column values of data tables has many data science applications such as schema matching, and data discovery and linking. For example, automatically extracted tables with missing headers can be filled by the predicted schema labels which significantly minimizes human effort. Furthermore, the predicted labels can reduce the impact of inconsistent names across multiple data tables. Understanding the connection between column values and contextual information is an important yet neglected aspect as previously proposed methods treat each column independently. In this paper, we propose a context-aware semantic labeling method using both the column values and context. Our new method is based on a new setting for semantic labeling, where we sequentially predict labels for an input table with missing headers. We incorporate both the values and context of each data column using the pre-trained contextualized language model, BERT, that has achieved significant improvements in multiple natural language processing tasks. To our knowledge, we are the first to successfully apply BERT to solve the semantic labeling task. We evaluate our approach using two real-world datasets from different domains, and we demonstrate substantial improvements in terms of evaluation metrics over state-of-the-art feature-based methods.