# On the Use of Default Parameter Settings in the Empirical Evaluation of Classification Algorithms

arXiv: https://arxiv.org/abs/1703.06777 · HTML: https://mlsanity.com/paper/1703.06777/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2017.

## Built on by (most cited first, 1 of 1)

- [Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization](https://mlsanity.com/paper/2204.10231/) (2022)

## Abstract (arXiv)

We demonstrate that, for a range of state-of-the-art machine learning algorithms, the differences in generalisation performance obtained using default parameter settings and using parameters tuned via cross-validation can be similar in magnitude to the differences in performance observed between state-of-the-art and uncompetitive learning systems. This means that fair and rigorous evaluation of new learning algorithms requires performance comparison against benchmark methods with best-practice model selection procedures, rather than using default parameter settings. We investigate the sensitivity of three key machine learning algorithms (support vector machine, random forest and rotation forest) to their default parameter settings, and provide guidance on determining sensible default parameter values for implementations of these algorithms. We also conduct an experimental comparison of these three algorithms on 121 classification problems and find that, perhaps surprisingly, rotation forest is significantly more accurate on average than both random forest and a support vector machine.