# X-Factor: Quality Is a Dataset-Intrinsic Property

arXiv: https://arxiv.org/abs/2505.22813 · HTML: https://mlsanity.com/paper/2505.22813/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Beyond neural scaling laws: beating power law scaling via data pruning](https://mlsanity.com/paper/2206.14486/) (2022)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt](https://mlsanity.com/paper/2206.07137/) (2022)
- [DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning](https://mlsanity.com/paper/2204.08499/) (2022)
- [A Solvable Model of Neural Scaling Laws](https://mlsanity.com/paper/2210.16859/) (2022)
- [A Dynamical Model of Neural Scaling Laws](https://mlsanity.com/paper/2402.01092/) (2024)
- [RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning](https://mlsanity.com/paper/2106.07760/) (2021)
- [D2 Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning](https://mlsanity.com/paper/2310.07931/) (2023)
- [A Simple Model of Inference Scaling Laws](https://mlsanity.com/paper/2410.16377/) (2024)
- [Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data](https://mlsanity.com/paper/2111.02275/) (2021)
- [Bayesian Active Learning with Fully Bayesian Gaussian Processes](https://mlsanity.com/paper/2205.10186/) (2022)
- [Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learning](https://mlsanity.com/paper/2205.15458/) (2022)
- [Neural Scaling Laws Rooted in the Data Distribution](https://mlsanity.com/paper/2412.07942/) (2024)
- [ELFS: Label-Free Coreset Selection with Proxy Training Dynamics](https://mlsanity.com/paper/2406.04273/) (2024)
- [Zero-Shot Coreset Selection via Iterative Subspace Sampling](https://mlsanity.com/paper/2411.15349/) (2024)
- [Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers](https://mlsanity.com/paper/2110.06990/) (2021)
- [Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization](https://mlsanity.com/paper/2504.14200/) (2025)
- [Neural Scaling Laws From Large-N Field Theory: Solvable Model Beyond the Ridgeless Limit](https://mlsanity.com/paper/2405.19398/) (2024)
- [Structural-Entropy-Based Sample Selection for Efficient and Effective Learning](https://mlsanity.com/paper/2410.02268/) (2024)
- [Beyond Size and Class Balance: Alpha as a New Dataset Quality Metric for Deep Learning](https://mlsanity.com/paper/2407.15724/) (2024)

## Built on by (most cited first, 3 of 3)

- [MH-1M: A 1.34 Million-Sample Comprehensive Multi-Feature Android Malware Dataset for Machine Learning, Deep Learning, Large Language Models, and Threat Intelligence Research](https://mlsanity.com/paper/2511.00342/) (2025)
- [Which Similarity-Sensitive Entropy (Sentropy)?](https://mlsanity.com/paper/2511.03849/) (2025)
- [Efficient, Validation-Free Intrinsic Quality Estimation for Large-Scale Face Recognition Datasets](https://mlsanity.com/paper/2605.29720/) (2026)

## Abstract (arXiv)

In the universal quest to optimize machine-learning classifiers, three factors -- model architecture, dataset size, and class balance -- have been shown to influence test-time performance but do not fully account for it. Previously, evidence was presented for an additional factor that can be referred to as dataset quality, but it was unclear whether this was actually a joint property of the dataset and the model architecture, or an intrinsic property of the dataset itself. If quality is truly dataset-intrinsic and independent of model architecture, dataset size, and class balance, then the same datasets should perform better (or worse) regardless of these other factors. To test this hypothesis, here we create thousands of datasets, each controlled for size and class balance, and use them to train classifiers with a wide range of architectures, from random forests and support-vector machines to deep networks. We find that classifier performance correlates strongly by subset across architectures (R2=0.79), supporting quality as an intrinsic property of datasets independent of dataset size and class balance and of model architecture. Digging deeper, we find that dataset quality appears to be an emergent property of something more fundamental: the quality of datasets' constituent classes. Thus, quality joins size, class balance, and model architecture as an independent correlate of performance and a separate target for optimizing machine-learning-based classification.