# Benchmarking as Empirical Standard in Software Engineering Research

arXiv: https://arxiv.org/abs/2105.00272 · HTML: https://mlsanity.com/paper/2105.00272/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2021.

## Built on by (most cited first, 2 of 2)

- [Innamark: A Whitespace Replacement Information-Hiding Method](https://mlsanity.com/paper/2502.12710/) (2025)
- [Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice](https://mlsanity.com/paper/2203.15414/) (2022)

## Abstract (arXiv)

In empirical software engineering, benchmarks can be used for comparing different methods, techniques and tools. However, the recent ACM SIGSOFT Empirical Standards for Software Engineering Research do not include an explicit checklist for benchmarking. In this paper, we discuss benchmarks for software performance and scalability evaluation as example research areas in software engineering, relate benchmarks to some other empirical research methods, and discuss the requirements on benchmarks that may constitute the basis for a checklist of a benchmarking standard for empirical software engineering research.