# PLM4NDV: Minimizing Data Access for Number of Distinct Values Estimation with Pre-trained Language Models

arXiv: https://arxiv.org/abs/2504.00608 · HTML: https://mlsanity.com/paper/2504.00608/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 92 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020)
- [GitTables: A Large-Scale Corpus of Relational Tables](https://mlsanity.com/paper/2106.07258/) (2021)
- [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021)
- [D-Bot: Database Diagnosis System using Large Language Models](https://mlsanity.com/paper/2312.01454/) (2023)
- [Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generation](https://mlsanity.com/paper/2306.08891/) (2023)
- [TabLib: A Dataset of 627M Tables with Context](https://mlsanity.com/paper/2310.07875/) (2023)
- [ALECE: An Attention-based Learned Cardinality Estimator for SPJ Queries on Dynamic Workloads (Extended)](https://mlsanity.com/paper/2310.05349/) (2023)
- [FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Language](https://mlsanity.com/paper/2310.17306/) (2023)
- [Approximate Profile Maximum Likelihood](https://mlsanity.com/paper/1712.07177/) (2017)
- [Unified Sample-Optimal Property Estimation in Near-Linear Time](https://mlsanity.com/paper/1911.03105/) (2019)
- [Learning to be a Statistician: Learned Estimator for Number of Distinct Values](https://mlsanity.com/paper/2202.02800/) (2022)
- [Cardinality Estimation in a Virtualized Network Device Using Online Machine Learning](https://mlsanity.com/paper/1903.05728/) (2019)
- [UltraLogLog: A Practical and More Space-Efficient Alternative to HyperLogLog for Approximate Distinct Counting](https://mlsanity.com/paper/2308.16862/) (2023)
- [AdaNDV: Adaptive Number of Distinct Value Estimation via Learning to Select and Fuse Estimators](https://mlsanity.com/paper/2502.16190/) (2025)

## Abstract (arXiv)

Number of Distinct Values (NDV) estimation of a multiset/column is a basis for many data management tasks, especially within databases. Despite decades of research, most existing methods require either a significant amount of samples through uniform random sampling or access to the entire column to produce estimates, leading to substantial data access costs and potentially ineffective estimations in scenarios with limited data access. In this paper, we propose leveraging semantic information, i.e., schema, to address these challenges. The schema contains rich semantic information that can benefit the NDV estimation. To this end, we propose PLM4NDV, a learned method incorporating Pre-trained Language Models (PLMs) to extract semantic schema information for NDV estimation. Specifically, PLM4NDV leverages the semantics of the target column and the corresponding table to gain a comprehensive understanding of the column's meaning. By using the semantics, PLM4NDV reduces data access costs, provides accurate NDV estimation, and can even operate effectively without any data access. Extensive experiments on a large-scale real-world dataset demonstrate the superiority of PLM4NDV over baseline methods. Our code is available at https://github.com/bytedance/plm4ndv.