mlsanityData to 7 Oct 2026

Is Hyper-Parameter Optimization Different for Software Analytics?

arXiv 2401.09622 ↗First posted 17 Jan 2024cs.SE cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Yes. SE data can have "smoother" boundaries between classes (compared to traditional AI data sets). To be more precise, the magnitude of the second derivative of the loss function found in SE data is typically much smaller. A new hyper-parameter optimizer, called SMOOTHIE, can exploit this idiosyncrasy of SE data. We compare SMOOTHIE and a state-of-the-art AI hyper-parameter optimizer on three tasks: (a) GitHub issue lifetime prediction (b) detecting static code warnings false alarm; (c) defect prediction. For completeness, we also show experiments on some standard AI datasets. SMOOTHIE runs faster and predicts better on the SE data--but ties on non-SE data with the AI tool. Hence we conclude that SE data can be different to other kinds of data; and those differences mean that we should use different kinds of algorithms for our data. To support open science and other researchers working in this area, all our scripts and datasets are available on-line at https://github.com/yrahul3910/smoothness-hpo/.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 75 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

34 of 75 references.

References · 34

  1. 2021Convex optimizationcited by 2,479
  2. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  3. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  4. 2017Exploring Generalization in Deep Learningcited by 595
  5. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  6. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  7. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  8. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  9. 2017Three Factors Influencing Minima in SGDcited by 270
  10. 2017The Marginal Value of Adaptive Gradient Methods in Machine Learningcited by 248
  11. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
  12. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 201
22 more
  1. 2019Adaptive Gradient Methods with Dynamic Bound of Learning Ratecited by 181
  2. 2020On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practicecited by 181
  3. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  4. 2020Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learningcited by 125
  5. 2017Improving Generalization Performance by Switching from Adam to SGDcited by 121
  6. 2020Hyper-Parameter Optimization: A Review of Algorithms and Applicationscited by 114
  7. 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
  8. 2021What Happens after SGD Reaches Zero Loss? --A Mathematical Frameworkcited by 106
  9. 2022When Do Flat Minima Optimizers Work?cited by 78
  10. 2017Hyperparameter Importance Across Datasetscited by 62
  11. 2021YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimizationcited by 61
  12. 2023The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descentcited by 44
  13. 2018DeepTriage: Exploring the Effectiveness of Deep Learning for Bug Triagingcited by 20
  14. 2016Tuning for Software Analytics: is it Really Necessary?cited by 18
  15. 2022Detecting False Alarms from Automatic Static Analysis Tools: How Far are We?cited by 16
  16. 2018500+ Times Faster Than Deep Learning (A Case Study Exploring Faster Methods for Text Mining StackOverflow)cited by 13
  17. 2022The activity-weight duality in feed forward neural networks: The geometric determinants of generalizationcited by 12
  18. 2019How to "DODGE" Complex Software Analytics?cited by 11
  19. 2020Simpler Hyperparameter Optimization for Software Analytics: Why, How, When?cited by 5
  20. 2022Flatter, faster: scaling momentum for optimal speedup of SGDcited by 5
  21. 2021Old but Gold: Reconsidering the value of feedforward learners for software analyticscited by 1
  22. 2022How to Find Actionable Static Analysis Warnings: A Case Study with FindBugscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Evaluating Useful Surrogate Models for Configuration Tuning Beyond Accuracy: A Fitness Landscape Analysis Perspectivecited by 0