# Predictability of Machine Learning Algorithms and Related Feature Extraction Techniques

arXiv: https://arxiv.org/abs/2305.00449 · HTML: https://mlsanity.com/paper/2305.00449/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 83 references that are in the map.
- First posted on arXiv 30 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [Hyperparameter Importance Across Datasets](https://mlsanity.com/paper/1710.04725/) (2017)
- [Benchmarking Automatic Machine Learning Frameworks](https://mlsanity.com/paper/1808.06492/) (2018)
- [Endgame Analysis of Dou Shou Qi](https://mlsanity.com/paper/1604.07312/) (2016)
- [Efficient L1-Norm Principal-Component Analysis via Bit Flipping](https://mlsanity.com/paper/1610.01959/) (2016)
- [Perceptron Mistake Bounds](https://mlsanity.com/paper/1305.0208/) (2013)
- [Optimization of data-driven filterbank for automatic speaker verification](https://mlsanity.com/paper/2007.10729/) (2020)
- [Extracting more from boosted decision trees: A high energy physics case study](https://mlsanity.com/paper/2001.06033/) (2020)

## Abstract (arXiv)

This thesis designs a prediction system based on matrix factorization to predict the classification accuracy of a specific model on a particular dataset. In this thesis, we conduct comprehensive empirical research on more than fifty datasets that we collected from the openml website. We study the performance prediction of three fundamental machine learning algorithms, namely, random forest, XGBoost, and MultiLayer Perceptron(MLP). In particular, we obtain the following results: 1. Predictability of fine-tuned models using coarse-tuned variants. 2. Predictability of MLP using feature extraction techniques. 3. Predict model performance using implicit feedback.