# Rethinking Performance Analysis for Configurable Software Systems: A Case Study from a Fitness Landscape Perspective

arXiv: https://arxiv.org/abs/2412.16888 · HTML: https://mlsanity.com/paper/2412.16888/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 91 references that are in the map.
- First posted on arXiv 22 Dec 2024.

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search](https://mlsanity.com/paper/2007.00708/) (2020)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [Finding Faster Configurations using FLASH](https://mlsanity.com/paper/1801.02175/) (2018)
- [Multi-Objectivizing Software Configuration Tuning (for a single performance concern)](https://mlsanity.com/paper/2106.01331/) (2021)
- [Do Performance Aspirations Matter for Guiding Software Configuration Tuning?](https://mlsanity.com/paper/2301.03290/) (2023)
- [Improving Protein Optimization with Smoothed Fitness Landscapes](https://mlsanity.com/paper/2307.00494/) (2023)
- [Transfer Learning for Performance Modeling of Configurable Systems: An Exploratory Analysis](https://mlsanity.com/paper/1709.02280/) (2017)
- [Meta-Surrogate Benchmarking for Hyperparameter Optimization](https://mlsanity.com/paper/1905.12982/) (2019)
- [Predicting Configuration Performance in Multiple Environments with Sequential Meta-learning](https://mlsanity.com/paper/2402.03183/) (2024)
- [Transfer Learning for Improving Model Predictions in Highly Configurable Software](https://mlsanity.com/paper/1704.00234/) (2017)
- [White-Box Analysis over Machine Learning: Modeling Performance of Configurable Systems](https://mlsanity.com/paper/2101.05362/) (2021)
- [Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes](https://mlsanity.com/paper/2203.15988/) (2022)
- [White-Box Performance-Influence Models: A Profiling and Learning Approach](https://mlsanity.com/paper/2102.06395/) (2021)
- [Using Bad Learners to find Good Configurations](https://mlsanity.com/paper/1702.05701/) (2017)
- [Faster Discovery of Faster System Configurations with Spectral Learning](https://mlsanity.com/paper/1701.08106/) (2017)
- [HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems](https://mlsanity.com/paper/2204.03931/) (2022)
- [On the Hyperparameter Loss Landscapes of Machine Learning Models: An Exploratory Study](https://mlsanity.com/paper/2311.14014/) (2023)
- [Learning from Very Little Data: On the Value of Landscape Analysis for Predicting Software Project Health](https://mlsanity.com/paper/2301.06577/) (2023)
- [Towards the Inferrence of Structural Similarity of Combinatorial Landscapes](https://mlsanity.com/paper/2312.02720/) (2023)

## Built on by (most cited first, 3 of 3)

- [Revealing Domain-Spatiality Patterns for Configuration Tuning: Domain Knowledge Meets Fitness Landscapes](https://mlsanity.com/paper/2603.19897/) (2026)
- [Evaluating Useful Surrogate Models for Configuration Tuning Beyond Accuracy: A Fitness Landscape Analysis Perspective](https://mlsanity.com/paper/2509.21945/) (2025)
- [When to Use Which? Benchmarking Optimisers for Configurable Systems under Varying Budgets](https://mlsanity.com/paper/2607.16476/) (2026)

## Abstract (arXiv)

Modern software systems are often highly configurable to tailor varied requirements from diverse stakeholders. Understanding the mapping between configurations and the desired performance attributes plays a fundamental role in advancing the controllability and tuning of the underlying system, yet has long been a dark hole of knowledge due to its black-box nature. While there have been previous efforts in performance analysis for these systems, they analyze the configurations as isolated data points without considering their inherent spatial relationships. This renders them incapable of interrogating many important aspects of the configuration space like local optima. In this work, we advocate a novel perspective to rethink performance analysis -- modeling the configuration space as a structured ``landscape''. To support this proposition, we designed our, an open-source, graph data mining empowered fitness landscape analysis (FLA) framework. By applying this framework to 86M benchmarked configurations from 32 running workloads of 3 real-world systems, we arrived at 6 main findings, which together constitute a holistic picture of the landscape topography, with thorough discussions about their implications on both configuration tuning and performance modeling.