# Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes

arXiv: https://arxiv.org/abs/2203.15988 · HTML: https://mlsanity.com/paper/2203.15988/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 56 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 15 of 15)

- [Finding Faster Configurations using FLASH](https://mlsanity.com/paper/1801.02175/) (2018)
- [Multi-Objectivizing Software Configuration Tuning (for a single performance concern)](https://mlsanity.com/paper/2106.01331/) (2021)
- [MMO: Meta Multi-Objectivization for Software Configuration Tuning](https://mlsanity.com/paper/2112.07303/) (2021)
- [FEMOSAA: Feature Guided and Knee Driven Multi-Objective Optimization for Self-Adaptive Software](https://mlsanity.com/paper/1608.08933/) (2016)
- [A Survey and Taxonomy of Self-Aware and Self-Adaptive Cloud Autoscaling Systems](https://mlsanity.com/paper/1609.03590/) (2016)
- [An Uncertainty-Aware Approach to Optimal Configuration of Stream Processing Systems](https://mlsanity.com/paper/1606.06543/) (2016)
- [Self-Adaptive Trade-off Decision Making for Autoscaling Cloud-Based Services](https://mlsanity.com/paper/1608.05917/) (2016)
- [How to Evaluate Solutions in Pareto-based Search-Based Software Engineering? A Critical Review and Methodological Guidance](https://mlsanity.com/paper/2002.09040/) (2020)
- [Lifelong Dynamic Optimization for Self-Adaptive Systems: Fact or Fiction?](https://mlsanity.com/paper/2201.07096/) (2022)
- [Hyperparameter Optimization for Effort Estimation](https://mlsanity.com/paper/1805.00336/) (2018)
- [Learning to Sample: an Active Learning Framework](https://mlsanity.com/paper/1909.03585/) (2019)
- [Synergizing Domain Expertise with Self-Awareness in Software Systems: A Patternized Architecture Guideline](https://mlsanity.com/paper/2001.07076/) (2020)
- [Using Bad Learners to find Good Configurations](https://mlsanity.com/paper/1702.05701/) (2017)
- [Online QoS Modeling in the Cloud: A Hybrid and Adaptive Multi-Learners Approach](https://mlsanity.com/paper/1504.03961/) (2015)
- [Binarsity: a penalization for one-hot encoded features in linear supervised learning](https://mlsanity.com/paper/1703.08619/) (2017)

## Built on by (most cited first, 10 of 10)

- [Do Performance Aspirations Matter for Guiding Software Configuration Tuning?](https://mlsanity.com/paper/2301.03290/) (2023)
- [Predicting Configuration Performance in Multiple Environments with Sequential Meta-learning](https://mlsanity.com/paper/2402.03183/) (2024)
- [Predicting Software Performance with Divide-and-Learn](https://mlsanity.com/paper/2306.06651/) (2023)
- [Adapting Multi-objectivized Software Configuration Tuning](https://mlsanity.com/paper/2404.04744/) (2024)
- [Deep Configuration Performance Learning: A Systematic Survey and Taxonomy](https://mlsanity.com/paper/2403.03322/) (2024)
- [Dividable Configuration Performance Learning](https://mlsanity.com/paper/2409.07629/) (2024)
- [Rethinking Performance Analysis for Configurable Software Systems: A Case Study from a Fitness Landscape Perspective](https://mlsanity.com/paper/2412.16888/) (2024)
- [Evaluating Useful Surrogate Models for Configuration Tuning Beyond Accuracy: A Fitness Landscape Analysis Perspective](https://mlsanity.com/paper/2509.21945/) (2025)
- [Multifaceted Hierarchical Report Identification for Non-Functional Bugs in Deep Learning Frameworks](https://mlsanity.com/paper/2210.01855/) (2022)
- [Hardness, Structural Knowledge, and Opportunity: An Analytical Framework for Modular Performance Modeling](https://mlsanity.com/paper/2509.11000/) (2025)

## Abstract (arXiv)

Learning and predicting the performance of a configurable software system helps to provide better quality assurance. One important engineering decision therein is how to encode the configuration into the model built. Despite the presence of different encoding schemes, there is still little understanding of which is better and under what circumstances, as the community often relies on some general beliefs that inform the decision in an ad-hoc manner. To bridge this gap, in this paper, we empirically compared the widely used encoding schemes for software performance learning, namely label, scaled label, and one-hot encoding. The study covers five systems, seven models, and three encoding schemes, leading to 105 cases of investigation. Our key findings reveal that: (1) conducting trial-and-error to find the best encoding scheme in a case by case manner can be rather expensive, requiring up to 400+ hours on some models and systems; (2) the one-hot encoding often leads to the most accurate results while the scaled label encoding is generally weak on accuracy over different models; (3) conversely, the scaled label encoding tends to result in the fastest training time across the models/systems while the one-hot encoding is the slowest; (4) for all models studied, label and scaled label encoding often lead to relatively less biased outcomes between accuracy and training time, but the paired model varies according to the system. We discuss the actionable suggestions derived from our findings, hoping to provide a better understanding of this topic for the community. To promote open science, the data and code of this work can be publicly accessed at https://github.com/ideas-labo/MSR2022-encoding-study.