# Fuzzy hyperparameters update in a second order optimization

arXiv: https://arxiv.org/abs/2403.15416 · HTML: https://mlsanity.com/paper/2403.15416/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Builds on (most cited first, 7 of 7)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning](https://mlsanity.com/paper/2006.00719/) (2020)
- [Hyper-Parameter Optimization: A Review of Algorithms and Applications](https://mlsanity.com/paper/2003.05689/) (2020)
- [A Survey of Optimization Methods from a Machine Learning Perspective](https://mlsanity.com/paper/1906.06821/) (2019)
- [Sub-sampled Cubic Regularization for Non-convex Optimization](https://mlsanity.com/paper/1705.05933/) (2017)
- [Block-diagonal Hessian-free Optimization for Training Neural Networks](https://mlsanity.com/paper/1712.07296/) (2017)
- [Approximating the diagonal of a Hessian: which sample set of points should be used](https://mlsanity.com/paper/2104.11821/) (2021)

## Abstract (arXiv)

This research will present a hybrid approach to accelerate convergence in a second order optimization. An online finite difference approximation of the diagonal Hessian matrix will be introduced, along with fuzzy inferencing of several hyperparameters. Competitive results have been achieved