mlsanityData to 7 Oct 2026

Reproducibility in Evolutionary Computation

arXiv 2102.03380 ↗First posted 5 Feb 2021cs.AI cs.NE math.OCcited by 12

Not on the map. Why?

Abstract (arXiv)

Experimental studies are prevalent in Evolutionary Computation (EC), and concerns about the reproducibility and replicability of such studies have increased in recent times, reflecting similar concerns in other scientific fields. In this article, we discuss, within the context of EC, the different types of reproducibility and suggest a classification that refines the badge system of the Association of Computing Machinery (ACM) adopted by ACM Transactions on Evolutionary Learning and Optimization (https://dlnext.acm.org/journal/telo). We identify cultural and technical obstacles to reproducibility in the EC field. Finally, we provide guidelines and suggest tools that may help to overcome some of these reproducibility obstacles.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 5 of its 77 references that are in the map. First posted on arXiv 5 Feb 2021.

Builds on

5 of 77 references.

References · 5

  1. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  2. 2017On the State of the Art of Evaluation in Neural Language Modelscited by 103
  3. 2020Benchmarking in Optimization: Best Practice and Open Issuescited by 35
  4. 2015The Configurable SAT Solver Challenge (CSSC)cited by 9
  5. 2019Sample size calculations for the experimental comparison of multiple algorithms on multiple problem instancescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 1; 2025: 3; 2026: 2.

12 papers.

  1. 2025BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristicscited by 9
  2. 2024Metaheuristics and Large Language Models Join Forces: Toward an Integrated Optimization Approachcited by 6
  3. 2023Using Automated Algorithm Configuration for Parameter Controlcited by 5
  4. 2023Using Knowledge Graphs for Performance Prediction of Modular Optimization Algorithmscited by 4
  5. 2023Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gymcited by 3
  6. 2025IterMask3D: Unsupervised Anomaly Detection and Segmentation with Test-Time Iterative Mask Refinement in 3D Brain MRcited by 3
  7. 2022The importance of being constrained: dealing with infeasible solutions in Differential Evolution and beyondcited by 1
  8. 2026Assessing Reproducibility in Evolutionary Computation: A Case Study using Human- and LLM-based Assessmentcited by 1
  9. 2022Reliable validation of Reinforcement Learning Benchmarkscited by 0
  10. 2022OPTION: OPTImization Algorithm Benchmarking ONtologycited by 0
  11. 2025Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Networkcited by 0
  12. 2026Metric Aggregation Divergence: A Hidden Validity Threat in Agent-Based Policy Optimization and a Contractual Remedycited by 0