# Perona: Robust Infrastructure Fingerprinting for Resource-Efficient Big Data Analytics

arXiv: https://arxiv.org/abs/2211.08227 · HTML: https://mlsanity.com/paper/2211.08227/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 14 of 14)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Topology Adaptive Graph Convolutional Networks](https://mlsanity.com/paper/1710.10370/) (2017)
- [TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud via Sub-Sampling](https://mlsanity.com/paper/2011.04726/) (2020)
- [Low-Level Augmented Bayesian Optimization for Finding the Best Cloud VM](https://mlsanity.com/paper/1712.10081/) (2017)
- [C3O: Collaborative Cluster Configuration Optimization for Distributed Data Processing in Public Clouds](https://mlsanity.com/paper/2107.13317/) (2021)
- [Bellamy: Reusing Performance Models for Distributed Dataflow Jobs Across Contexts](https://mlsanity.com/paper/2107.13921/) (2021)
- [Enel: Context-Aware Dynamic Scaling of Distributed Dataflow Jobs using Graph Propagation](https://mlsanity.com/paper/2108.12211/) (2021)
- [Tarema: Adaptive Resource Allocation for Scalable Scientific Workflows in Heterogeneous Clusters](https://mlsanity.com/paper/2111.05167/) (2021)
- [Lotaru: Locally Estimating Runtimes of Scientific Workflow Tasks in Heterogeneous Clusters](https://mlsanity.com/paper/2205.11181/) (2022)
- [Reshi: Recommending Resources for Scientific Workflow Tasks on Heterogeneous Infrastructures](https://mlsanity.com/paper/2208.07905/) (2022)

## Built on by (most cited first, 1 of 1)

- [Karasu: A Collaborative Approach to Efficient Cluster Configuration for Big Data Analytics](https://mlsanity.com/paper/2308.11792/) (2023)

## Abstract (arXiv)

Choosing a good resource configuration for big data analytics applications can be challenging, especially in cloud environments. Automated approaches are desirable as poor decisions can reduce performance and raise costs. The majority of existing automated approaches either build performance models from previous workload executions or conduct iterative resource configuration profiling until a near-optimal solution has been found. In doing so, they only obtain an implicit understanding of the underlying infrastructure, which is difficult to transfer to alternative infrastructures and, thus, profiling and modeling insights are not sustained beyond very specific situations. We present Perona, a novel approach to robust infrastructure fingerprinting for usage in the context of big data analytics. Perona employs common sets and configurations of benchmarking tools for target resources, so that resulting benchmark metrics are directly comparable and ranking is enabled. Insignificant benchmark metrics are discarded by learning a low-dimensional representation of the input metric vector, and previous benchmark executions are taken into consideration for context-awareness as well, allowing to detect resource degradation. We evaluate our approach both on data gathered from our own experiments as well as within related works for resource configuration optimization, demonstrating that Perona captures the characteristics from benchmark runs in a compact manner and produces representations that can be used directly.