# Composite Optimization Algorithms for Sigmoid Networks

arXiv: https://arxiv.org/abs/2303.00589 · HTML: https://mlsanity.com/paper/2303.00589/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 19 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 3 of 3)

- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [The Marginal Value of Adaptive Gradient Methods in Machine Learning](https://mlsanity.com/paper/1705.08292/) (2017)
- [Adaptive Regularization in Convex Composite Optimization for Variational Imaging Problems](https://mlsanity.com/paper/1609.02356/) (2016)

## Abstract (arXiv)

In this paper, we use composite optimization algorithms to solve sigmoid networks. We equivalently transfer the sigmoid networks to a convex composite optimization and propose the composite optimization algorithms based on the linearized proximal algorithms and the alternating direction method of multipliers. Under the assumptions of the weak sharp minima and the regularity condition, the algorithm is guaranteed to converge to a globally optimal solution of the objective function even in the case of non-convex and non-smooth problems. Furthermore, the convergence results can be directly related to the amount of training data and provide a general guide for setting the size of sigmoid networks. Numerical experiments on Franke's function fitting and handwritten digit recognition show that the proposed algorithms perform satisfactorily and robustly.