# Performance evaluation and hyperparameter tuning of statistical and machine-learning models using spatial data

arXiv: https://arxiv.org/abs/1803.11266 · HTML: https://mlsanity.com/paper/1803.11266/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2018.

## Built on by (most cited first, 11 of 11)

- [Species Distribution Modeling for Machine Learning Practitioners: A Review](https://mlsanity.com/paper/2107.10400/) (2021)
- [Channel Selected Stratified Nested Cross Validation for Clinically Relevant EEG Based Parkinsons Disease Detection](https://mlsanity.com/paper/2601.05276/) (2025)
- [A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques](https://mlsanity.com/paper/2302.08571/) (2023)
- [Foundation for unbiased cross-validation of spatio-temporal models for species distribution modeling](https://mlsanity.com/paper/2502.03480/) (2025)
- [Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices](https://mlsanity.com/paper/2506.04553/) (2025)
- [Gradient Boosting for Spatial Regression Models with Autoregressive Disturbances](https://mlsanity.com/paper/2506.13682/) (2025)
- [Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization](https://mlsanity.com/paper/2107.08761/) (2021)
- [Machine Learning to Predict the Antimicrobial Activity of Cold Atmospheric Plasma-Activated Liquids](https://mlsanity.com/paper/2207.12478/) (2022)
- [Making informed decisions in cutting tool maintenance in milling: A KNN-based model agnostic approach](https://mlsanity.com/paper/2310.14629/) (2023)
- [Clutter Classification Using Deep Learning in Multiple Stages](https://mlsanity.com/paper/2408.04407/) (2024)
- [From Players to Champions: A Generalizable Machine Learning Approach for Match Outcome Prediction with Insights from the FIFA World Cup](https://mlsanity.com/paper/2505.01902/) (2025)

## Abstract (arXiv)

Machine-learning algorithms have gained popularity in recent years in the field of ecological modeling due to their promising results in predictive performance of classification problems. While the application of such algorithms has been highly simplified in the last years due to their well-documented integration in commonly used statistical programming languages such as R, there are several practical challenges in the field of ecological modeling related to unbiased performance estimation, optimization of algorithms using hyperparameter tuning and spatial autocorrelation. We address these issues in the comparison of several widely used machine-learning algorithms such as Boosted Regression Trees (BRT), k-Nearest Neighbor (WKNN), Random Forest (RF) and Support Vector Machine (SVM) to traditional parametric algorithms such as logistic regression (GLM) and semi-parametric ones like generalized additive models (GAM). Different nested cross-validation methods including hyperparameter tuning methods are used to evaluate model performances with the aim to receive bias-reduced performance estimates. As a case study the spatial distribution of forest disease Diplodia sapinea in the Basque Country in Spain is investigated using common environmental variables such as temperature, precipitation, soil or lithology as predictors. Results show that GAM and RF (mean AUROC estimates 0.708 and 0.699) outperform all other methods in predictive accuracy. The effect of hyperparameter tuning saturates at around 50 iterations for this data set. The AUROC differences between the bias-reduced (spatial cross-validation) and overoptimistic (non-spatial cross-validation) performance estimates of the GAM and RF are 0.167 (24%) and 0.213 (30%), respectively. It is recommended to also use spatial partitioning for cross-validation hyperparameter tuning of spatial data.