# Do Performance Aspirations Matter for Guiding Software Configuration Tuning?

arXiv: https://arxiv.org/abs/2301.03290 · HTML: https://mlsanity.com/paper/2301.03290/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 101 references that are in the map.
- First posted on arXiv 9 Jan 2023.

## Builds on (most cited first, 16 of 16)

- [Finding Faster Configurations using FLASH](https://mlsanity.com/paper/1801.02175/) (2018)
- [Multi-Objectivizing Software Configuration Tuning (for a single performance concern)](https://mlsanity.com/paper/2106.01331/) (2021)
- [BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning](https://mlsanity.com/paper/1710.03439/) (2017)
- [FEMOSAA: Feature Guided and Knee Driven Multi-Objective Optimization for Self-Adaptive Software](https://mlsanity.com/paper/1608.08933/) (2016)
- [A Survey and Taxonomy of Self-Aware and Self-Adaptive Cloud Autoscaling Systems](https://mlsanity.com/paper/1609.03590/) (2016)
- [An Uncertainty-Aware Approach to Optimal Configuration of Stream Processing Systems](https://mlsanity.com/paper/1606.06543/) (2016)
- [Self-Adaptive Trade-off Decision Making for Autoscaling Cloud-Based Services](https://mlsanity.com/paper/1608.05917/) (2016)
- [How to Evaluate Solutions in Pareto-based Search-Based Software Engineering? A Critical Review and Methodological Guidance](https://mlsanity.com/paper/2002.09040/) (2020)
- [The Weights can be Harmful: Pareto Search versus Weighted Search in Multi-Objective Search-Based Software Engineering](https://mlsanity.com/paper/2202.03728/) (2022)
- [BiLO-CPDP: Bi-Level Programming for Automated Model Discovery in Cross-Project Defect Prediction](https://mlsanity.com/paper/2008.13489/) (2020)
- [Lifelong Dynamic Optimization for Self-Adaptive Systems: Fact or Fiction?](https://mlsanity.com/paper/2201.07096/) (2022)
- [Transfer Learning for Improving Model Predictions in Highly Configurable Software](https://mlsanity.com/paper/1704.00234/) (2017)
- [Unicorn: Reasoning about Configurable System Performance through the lens of Causality](https://mlsanity.com/paper/2201.08413/) (2022)
- [Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes](https://mlsanity.com/paper/2203.15988/) (2022)
- [DATESSO: Self-Adapting Service Composition with Debt-Aware Two Levels Constraint Reasoning](https://mlsanity.com/paper/2003.14377/) (2020)
- [Does Preference Always Help? A Holistic Study on Preference-Based Evolutionary Multi-Objective Optimisation Using Reference Points](https://mlsanity.com/paper/1909.13567/) (2019)

## Built on by (most cited first, 21 of 21)

- [Predicting Configuration Performance in Multiple Environments with Sequential Meta-learning](https://mlsanity.com/paper/2402.03183/) (2024)
- [Predicting Software Performance with Divide-and-Learn](https://mlsanity.com/paper/2306.06651/) (2023)
- [Adapting Multi-objectivized Software Configuration Tuning](https://mlsanity.com/paper/2404.04744/) (2024)
- [Distilled Lifelong Self-Adaptation for Configurable Systems](https://mlsanity.com/paper/2501.00840/) (2025)
- [Faster Configuration Performance Bug Testing with Neural Dual-level Prioritization](https://mlsanity.com/paper/2501.15392/) (2025)
- [Deep Configuration Performance Learning: A Systematic Survey and Taxonomy](https://mlsanity.com/paper/2403.03322/) (2024)
- [Dually Hierarchical Drift Adaptation for Online Configuration Performance Learning](https://mlsanity.com/paper/2507.08730/) (2025)
- [Revealing Domain-Spatiality Patterns for Configuration Tuning: Domain Knowledge Meets Fitness Landscapes](https://mlsanity.com/paper/2603.19897/) (2026)
- [CaRE: Finding Root Causes of Configuration Issues in Highly-Configurable Robots](https://mlsanity.com/paper/2301.07690/) (2023)
- [Dividable Configuration Performance Learning](https://mlsanity.com/paper/2409.07629/) (2024)
- [Rethinking Performance Analysis for Configurable Software Systems: A Case Study from a Fitness Landscape Perspective](https://mlsanity.com/paper/2412.16888/) (2024)
- [Learning Software Bug Reports: A Systematic Literature Review](https://mlsanity.com/paper/2507.04422/) (2025)
- [CURE: Simulation-Augmented Auto-Tuning in Robotics](https://mlsanity.com/paper/2402.05399/) (2024)
- [CoTune: Co-evolutionary Configuration Tuning](https://mlsanity.com/paper/2509.24694/) (2025)
- [Light over Heavy: Automated Performance Requirements Quantification with Linguistic Inducement](https://mlsanity.com/paper/2511.03421/) (2025)
- [Evaluating Useful Surrogate Models for Configuration Tuning Beyond Accuracy: A Fitness Landscape Analysis Perspective](https://mlsanity.com/paper/2509.21945/) (2025)
- [Conjecture and Inquiry: Quantifying Software Performance Requirements via Interactive Retrieval-Augmented Preference Elicitation](https://mlsanity.com/paper/2604.21380/) (2026)
- [When to Use Which? Benchmarking Optimisers for Configurable Systems under Varying Budgets](https://mlsanity.com/paper/2607.16476/) (2026)
- [Contexts Matter: An Empirical Study on Contextual Influence in Fairness Testing for Deep Learning Systems](https://mlsanity.com/paper/2408.06102/) (2024)
- [Human-Machine Co-Boosted Bug Report Identification with Mutualistic Neural Active Learning](https://mlsanity.com/paper/2604.18862/) (2026)
- [LLMSYS-HPOBench: Hyperparameter Optimization Benchmark Suite for Real-World LLM Systems](https://mlsanity.com/paper/2605.08305/) (2026)

## Abstract (arXiv)

Configurable software systems can be tuned for better performance. Leveraging on some Pareto optimizers, recent work has shifted from tuning for a single, time-related performance objective to two intrinsically different objectives that assess distinct performance aspects of the system, each with varying aspirations. Before we design better optimizers, a crucial engineering decision to make therein is how to handle the performance requirements with clear aspirations in the tuning process. For this, the community takes two alternative optimization models: either quantifying and incorporating the aspirations into the search objectives that guide the tuning, or not considering the aspirations during the search but purely using them in the later decision-making process only. However, despite being a crucial decision that determines how an optimizer can be designed and tailored, there is a rather limited understanding of which optimization model should be chosen under what particular circumstance, and why. In this paper, we seek to close this gap. Firstly, we do that through a review of over 426 papers in the literature and 14 real-world requirements datasets. Drawing on these, we then conduct a comprehensive empirical study that covers 15 combinations of the state-of-the-art performance requirement patterns, four types of aspiration space, three Pareto optimizers, and eight real-world systems/environments, leading to 1,296 cases of investigation. We found that (1) the realism of aspirations is the key factor that determines whether they should be used to guide the tuning; (2) the given patterns and the position of the realistic aspirations in the objective landscape are less important for the choice, but they do matter to the extents of improvement; (3) the available tuning budget can also influence the choice for unrealistic aspirations but it is insignificant under realistic ones.