# Methods for Quantifying Dataset Similarity: a Review, Taxonomy and Comparison

arXiv: https://arxiv.org/abs/2312.04078 · HTML: https://mlsanity.com/paper/2312.04078/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2023.

## Built on by (most cited first, 11 of 11)

- [BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation](https://mlsanity.com/paper/2502.04314/) (2025)
- [Accidental Vulnerability: Factors in Fine-Tuning that Shift Model Safeguards](https://mlsanity.com/paper/2505.16789/) (2025)
- [Quantifying Data Similarity Using Cross Learning](https://mlsanity.com/paper/2510.10866/) (2025)
- [Federated Learning for Heterogeneous Electronic Health Record Systems with Cost Effective Participant Selection](https://mlsanity.com/paper/2404.13318/) (2024)
- [Time-Series Forecasting in Smart Manufacturing Systems: An Experimental Evaluation of the State-of-the-art Algorithms](https://mlsanity.com/paper/2411.17499/) (2024)
- [Evaluating Facial Expression Recognition Datasets for Deep Learning: A Benchmark Study with Novel Similarity Metrics](https://mlsanity.com/paper/2503.20428/) (2025)
- [PromptSplit: Revealing Prompt-Level Disagreement in Generative Models](https://mlsanity.com/paper/2602.04009/) (2026)
- [Evaluation of Machine-generated Biomedical Images via A Tally-based Similarity Measure](https://mlsanity.com/paper/2503.22658/) (2025)
- [Efficient and Stable Multi-Dimensional Kolmogorov-Smirnov Distance](https://mlsanity.com/paper/2504.11299/) (2025)
- [Financial time series augmentation using transformer based GAN architecture](https://mlsanity.com/paper/2602.17865/) (2026)
- [Integrating Meta-Features with Knowledge Graph Embeddings for Meta-Learning](https://mlsanity.com/paper/2603.19888/) (2026)

## Abstract (arXiv)

Quantifying the similarity between datasets has widespread applications in statistics and machine learning. The performance of a predictive model on novel datasets, referred to as generalizability, depends on how similar the training and evaluation datasets are. Exploiting or transferring insights between similar datasets is a key aspect of meta-learning and transfer-learning. In simulation studies, the similarity between distributions of simulated datasets and real datasets, for which the performance of methods is assessed, is crucial. In two- or k-sample testing, it is checked, whether the underlying distributions of two or more datasets coincide. Extremely many approaches for quantifying dataset similarity have been proposed in the literature. We examine more than 100 methods and provide a taxonomy, classifying them into ten classes. In an extensive review of these methods the main underlying ideas, formal definitions, and important properties are introduced. We compare the 118 methods in terms of their applicability, interpretability, and theoretical properties, in order to provide recommendations for selecting an appropriate dataset similarity measure based on the specific goal of the dataset comparison and on the properties of the datasets at hand. An online tool facilitates the choice of the appropriate dataset similarity measure.