# Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation

arXiv: https://arxiv.org/abs/2109.05877 · HTML: https://mlsanity.com/paper/2109.05877/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 69 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Builds on (most cited first, 20 of 20)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Learned Cardinalities: Estimating Correlated Joins with Deep Learning](https://mlsanity.com/paper/1809.00677/) (2018)
- [An End-to-End Learning-based Cost Estimator](https://mlsanity.com/paper/1906.02560/) (2019)
- [Deep Unsupervised Cardinality Estimation](https://mlsanity.com/paper/1905.04278/) (2019)
- [FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation](https://mlsanity.com/paper/2011.09022/) (2020)
- [Flow-Loss: Learning Cardinality Estimates That Matter](https://mlsanity.com/paper/2101.04964/) (2021)
- [NeuroCard: One Cardinality Estimator for All Tables](https://mlsanity.com/paper/2006.08109/) (2020)
- [A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation](https://mlsanity.com/paper/2107.12295/) (2021)
- [DeepDB: Learn from Data, not from Queries!](https://mlsanity.com/paper/1909.00607/) (2019)
- [Are We Ready For Learned Cardinality Estimation?](https://mlsanity.com/paper/2012.06743/) (2020)
- [BayesCard: Revitilizing Bayesian Frameworks for Cardinality Estimation](https://mlsanity.com/paper/2012.14743/) (2020)
- [A Unified Transferable Model for ML-Enhanced DBMS](https://mlsanity.com/paper/2105.02418/) (2021)
- [Multi-Attribute Selectivity Estimation Using Deep Learning](https://mlsanity.com/paper/1903.09999/) (2019)
- [An Approach Based on Bayesian Networks for Query Selectivity Estimation](https://mlsanity.com/paper/1907.06295/) (2019)
- [FSPN: A New Class of Probabilistic Graphical Model](https://mlsanity.com/paper/2011.09020/) (2020)
- [Variable Skipping for Autoregressive Range Density Estimation](https://mlsanity.com/paper/2007.05572/) (2020)
- [How I Learned to Stop Worrying and Love Re-optimization](https://mlsanity.com/paper/1902.08291/) (2019)
- [Sum-Product Graphical Models](https://mlsanity.com/paper/1708.06438/) (2017)

## Built on by (most cited first, 30 of 30)

- [Lero: A Learning-to-Rank Query Optimizer](https://mlsanity.com/paper/2302.06873/) (2023)
- [MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2506.05587/) (2025)
- [FactorJoin: A New Cardinality Estimation Framework for Join Queries](https://mlsanity.com/paper/2212.05526/) (2022)
- [PRICE: A Pretrained Model for Cross-Database Cardinality Estimation](https://mlsanity.com/paper/2406.01027/) (2024)
- [CardBench: A Benchmark for Learned Cardinality Estimation in Relational Databases](https://mlsanity.com/paper/2408.16170/) (2024)
- [Kepler: Robust Learning for Faster Parametric Query Optimization](https://mlsanity.com/paper/2306.06798/) (2023)
- [Duet: efficient and scalable hybriD neUral rElation undersTanding](https://mlsanity.com/paper/2307.13494/) (2023)
- [A Practical Theory of Generalization in Selectivity Learning](https://mlsanity.com/paper/2409.07014/) (2024)
- [Glue: Adaptively Merging Single Table Cardinality to Estimate Join Query Size](https://mlsanity.com/paper/2112.03458/) (2021)
- [JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning](https://mlsanity.com/paper/2307.11704/) (2023)
- [Hype or Heuristic? Quantum Reinforcement Learning for Join Order Optimisation](https://mlsanity.com/paper/2405.07770/) (2024)
- [DistJoin: A Decoupled Join Cardinality Estimator based on Adaptive Neural Predicate Modulation](https://mlsanity.com/paper/2503.08994/) (2025)
- [Data-Agnostic Cardinality Learning from Imperfect Workloads](https://mlsanity.com/paper/2506.16007/) (2025)
- [Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet](https://mlsanity.com/paper/2507.19209/) (2025)
- [Is it Bigger than a Breadbox: Efficient Cardinality Estimation for Real World Workloads](https://mlsanity.com/paper/2510.03386/) (2025)
- [When Does q-error Predict Plan Regret? Three Regimes of Cardinality-Estimation Error](https://mlsanity.com/paper/2606.15600/) (2026)
- [Baihe: SysML Framework for AI-driven Databases](https://mlsanity.com/paper/2112.14460/) (2021)
- [Scardina: Scalable Join Cardinality Estimation by Multiple Density Estimators](https://mlsanity.com/paper/2303.18042/) (2023)
- [LearnedWMP: Workload Memory Prediction Using Distribution of Query Templates](https://mlsanity.com/paper/2401.12103/) (2024)
- [Reqo: A Comprehensive Learning-Based Cost Model for Robust and Explainable Query Optimization](https://mlsanity.com/paper/2501.17414/) (2025)
- [Optimize Cardinality Estimation Model Pretraining by Simplifying the Training Datasets](https://mlsanity.com/paper/2502.14350/) (2025)
- [Selective Use of Yannakakis' Algorithm to Improve Query Performance: Machine Learning to the Rescue](https://mlsanity.com/paper/2502.20233/) (2025)
- [ACE: A Cardinality Estimator for Set-Valued Queries](https://mlsanity.com/paper/2503.14929/) (2025)
- [Sketched Sum-Product Networks for Joins](https://mlsanity.com/paper/2506.14034/) (2025)
- [ZeroCard: Cardinality Estimation with Zero Dependence on Target Databases -- No Data, No Query, No Retraining](https://mlsanity.com/paper/2510.07983/) (2025)
- [CoLSE: A Lightweight and Robust Hybrid Learned Model for Single-Table Cardinality Estimation using Joint CDF](https://mlsanity.com/paper/2512.12624/) (2025)
- [MICRO: A Lightweight Middleware for Optimizing Cross-store Cross-model Graph-Relation Joins [Technical Report]](https://mlsanity.com/paper/2603.13835/) (2026)
- [Cardinality Estimation for High Dimensional Similarity Queries with Adaptive Bucket Probing](https://mlsanity.com/paper/2604.04603/) (2026)
- [Opti-Q: A Constraint-Based Optimization Framework for Multi-LLM Question Planning](https://mlsanity.com/paper/2607.22621/) (2026)
- [Fully Inductive Cardinality Estimation](https://mlsanity.com/paper/2607.28311/) (2026)

## Abstract (arXiv)

Cardinality estimation (CardEst) plays a significant role in generating high-quality query plans for a query optimizer in DBMS. In the last decade, an increasing number of advanced CardEst methods (especially ML-based) have been proposed with outstanding estimation accuracy and inference latency. However, there exists no study that systematically evaluates the quality of these methods and answer the fundamental problem: to what extent can these methods improve the performance of query optimizer in real-world settings, which is the ultimate goal of a CardEst method. In this paper, we comprehensively and systematically compare the effectiveness of CardEst methods in a real DBMS. We establish a new benchmark for CardEst, which contains a new complex real-world dataset STATS and a diverse query workload STATS-CEB. We integrate multiple most representative CardEst methods into an open-source database system PostgreSQL, and comprehensively evaluate their true effectiveness in improving query plan quality, and other important aspects affecting their applicability, ranging from inference latency, model size, and training time, to update efficiency and accuracy. We obtain a number of key findings for the CardEst methods, under different data and query settings. Furthermore, we find that the widely used estimation accuracy metric(Q-Error) cannot distinguish the importance of different sub-plan queries during query optimization and thus cannot truly reflect the query plan quality generated by CardEst methods. Therefore, we propose a new metric P-Error to evaluate the performance of CardEst methods, which overcomes the limitation of Q-Error and is able to reflect the overall end-to-end performance of CardEst methods. We have made all of the benchmark data and evaluation code publicly available at https://github.com/Nathaniel-Han/End-to-End-CardEst-Benchmark.