# Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy

arXiv: https://arxiv.org/abs/2006.11135 · HTML: https://mlsanity.com/paper/2006.11135/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2020.

## Built on by (most cited first, 27 of 27)

- [Towards Explainable Exploratory Landscape Analysis: Extreme Feature Selection for Classifying BBOB Functions](https://mlsanity.com/paper/2102.00736/) (2021)
- [Neural Exploratory Landscape Analysis for Meta-Black-Box-Optimization](https://mlsanity.com/paper/2408.10672/) (2024)
- [Deep-ELA: Deep Exploratory Landscape Analysis with Self-Supervised Pretrained Transformers for Single- and Multi-Objective Continuous Optimization Problems](https://mlsanity.com/paper/2401.01192/) (2024)
- [DynamoRep: Trajectory-Based Population Dynamics for Classification of Black-box Optimization Problems](https://mlsanity.com/paper/2306.05438/) (2023)
- [Automated Metaheuristic Algorithm Design with Autoregressive Learning](https://mlsanity.com/paper/2405.03419/) (2024)
- [Instance Generation for Meta-Black-Box Optimization through Latent Space Reverse Engineering](https://mlsanity.com/paper/2509.15810/) (2025)
- [The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variants](https://mlsanity.com/paper/2204.07431/) (2022)
- [PS-AAS: Portfolio Selection for Automated Algorithm Selection in Black-Box Optimization](https://mlsanity.com/paper/2310.10685/) (2023)
- [DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization](https://mlsanity.com/paper/2505.17866/) (2025)
- [Using Knowledge Graphs for Performance Prediction of Modular Optimization Algorithms](https://mlsanity.com/paper/2301.09876/) (2023)
- [Explainable Landscape Analysis in Automated Algorithm Performance Prediction](https://mlsanity.com/paper/2203.11828/) (2022)
- [TransOpt: Transformer-based Representation Learning for Optimization Problem Classification](https://mlsanity.com/paper/2311.18035/) (2023)
- [A Survey of Features Used for Representing Black-box Single-objective Continuous Optimization](https://mlsanity.com/paper/2406.06629/) (2024)
- [Automated Algorithm Selection for Radar Network Configuration](https://mlsanity.com/paper/2205.03670/) (2022)
- [Computational and Exploratory Landscape Analysis of the GKLS Generator](https://mlsanity.com/paper/2304.08913/) (2023)
- [Landscape-Aware Automated Algorithm Configuration using Multi-output Mixed Regression and Classification](https://mlsanity.com/paper/2409.01446/) (2024)
- [Tracing the Interactions of Modular CMA-ES Configurations Across Problem Landscapes](https://mlsanity.com/paper/2507.02331/) (2025)
- [Learning Where, What and How to Transfer: A Multi-Role Reinforcement Learning Approach for Evolutionary Multitasking](https://mlsanity.com/paper/2511.15199/) (2025)
- [Resource Planning for Hospitals Under Special Consideration of the COVID-19 Pandemic: Optimization and Sensitivity Analysis](https://mlsanity.com/paper/2105.07420/) (2021)
- [Landscape Analysis for Surrogate Models in the Evolutionary Black-Box Context](https://mlsanity.com/paper/2203.11315/) (2022)
- [OPTION: OPTImization Algorithm Benchmarking ONtology](https://mlsanity.com/paper/2211.11332/) (2022)
- [Hilbert curves for efficient exploratory landscape analysis neighbourhood sampling](https://mlsanity.com/paper/2408.00526/) (2024)
- [Reinforcement learning Based Automated Design of Differential Evolution Algorithm for Black-box Optimization](https://mlsanity.com/paper/2501.12881/) (2025)
- [Geometric Probing for Algorithm Selection in Continuous Black-Box Optimization](https://mlsanity.com/paper/2604.09095/) (2026)
- [Does Dimensionality Reduction via Random Projections Preserve Landscape Features?](https://mlsanity.com/paper/2604.13230/) (2026)
- [A New Approach to Characterising Optimisation Problems Using Programmatic Representation and Complexity Measures](https://mlsanity.com/paper/2608.08898/) (2026)
- [Reinforcement learning to choose optimizers](https://mlsanity.com/paper/2609.01811/) (2026)

## Abstract (arXiv)

Exploratory landscape analysis (ELA) supports supervised learning approaches for automated algorithm selection and configuration by providing sets of features that quantify the most relevant characteristics of the optimization problem at hand. In black-box optimization, where an explicit problem representation is not available, the feature values need to be approximated from a small number of sample points. In practice, uniformly sampled random point sets and Latin hypercube constructions are commonly used sampling strategies. In this work, we analyze how the sampling method and the sample size influence the quality of the feature value approximations and how this quality impacts the accuracy of a standard classification task. While, not unexpectedly, increasing the number of sample points gives more robust estimates for the feature values, to our surprise we find that the feature value approximations for different sampling strategies do not converge to the same value. This implies that approximated feature values cannot be interpreted independently of the underlying sampling strategy. As our classification experiments show, this also implies that the feature approximations used for training a classifier must stem from the same sampling strategy as those used for the actual classification tasks. As a side result we show that classifiers trained with feature values approximated by Sobol' sequences achieve higher accuracy than any of the standard sampling techniques. This may indicate improvement potential for ELA-trained machine learning models.