# Tuneful: An Online Significance-Aware Configuration Tuner for Big Data Analytics

arXiv: https://arxiv.org/abs/2001.08002 · HTML: https://mlsanity.com/paper/2001.08002/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2020.

## Built on by (most cited first, 5 of 5)

- [Bellamy: Reusing Performance Models for Distributed Dataflow Jobs Across Contexts](https://mlsanity.com/paper/2107.13921/) (2021)
- [Rover: An online Spark SQL tuning service via generalized transfer learning](https://mlsanity.com/paper/2302.04046/) (2023)
- [Enel: Context-Aware Dynamic Scaling of Distributed Dataflow Jobs using Graph Propagation](https://mlsanity.com/paper/2108.12211/) (2021)
- [Transfer-Learning-Based Autotuning Using Gaussian Copula](https://mlsanity.com/paper/2401.04669/) (2024)
- [Relation-Aware Bayesian Optimization of DBMS Configurations Guided by Affinity Scores](https://mlsanity.com/paper/2510.27145/) (2025)

## Abstract (arXiv)

Distributed analytics engines such as Spark are a common choice for processing extremely large datasets. However, finding good configurations for these systems remains challenging, with each workload potentially requiring a different setup to run optimally. Using suboptimal configurations incurs significant extra runtime costs. %Furthermore, Spark and similar platforms are gaining traction within data-scientists communities where awareness of such issues is relatively low. We propose Tuneful, an approach that efficiently tunes the configuration of in-memory cluster computing systems. Tuneful combines incremental Sensitivity Analysis and Bayesian optimization to identify near-optimal configurations from a high-dimensional search space, using a small number of executions. This setup allows the tuning to be done online, without any previous training. Our experimental results show that Tuneful reduces the search time for finding close-to-optimal configurations by 62% (at the median) when compared to existing state-of-the-art techniques. This means that the amortization of the tuning cost happens significantly faster, enabling practical tuning for new classes of workloads.