# Using Non-Lipschitz Signum-based Functions for Distributed Optimization and Machine Learning: Trade-off Between Con-vergence Rate and Optimality Gap

arXiv: https://arxiv.org/abs/2608.01220 · HTML: https://mlsanity.com/paper/2608.01220/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 67 references that are in the map.
- First posted on arXiv 2 Aug 2026.

## Builds on (most cited first, 21 of 21)

- [Distributed optimization over time-varying directed graphs](https://mlsanity.com/paper/1303.2289/) (2013)
- [Learning Theory for Distribution Regression](https://mlsanity.com/paper/1411.2066/) (2014)
- [A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimization](https://mlsanity.com/paper/2102.06752/) (2021)
- [Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functions](https://mlsanity.com/paper/1811.03982/) (2018)
- [S-ADDOPT: Decentralized stochastic first-order optimization over directed graphs](https://mlsanity.com/paper/2005.07785/) (2020)
- [An Analysis of Robustness of Non-Lipschitz Networks](https://mlsanity.com/paper/2010.06154/) (2020)
- [Distributed support-vector-machine over dynamic balanced directed networks](https://mlsanity.com/paper/2104.00399/) (2021)
- [Fast-Convergent Dynamics for Distributed Allocation of Resources Over Switching Sparse Networks with Quantized Communication Links](https://mlsanity.com/paper/2012.08181/) (2020)
- [Fast ADMM Algorithm for Distributed Optimization with Adaptive Penalty](https://mlsanity.com/paper/1506.08928/) (2015)
- [Single-Bit Consensus with Finite-Time Convergence: Theory and Applications](https://mlsanity.com/paper/2001.00141/) (2020)
- [Distributed Subgradient Projection Algorithm over Directed Graphs: Alternate Proof](https://mlsanity.com/paper/1706.07707/) (2017)
- [Accelerated Distributed Allocation](https://mlsanity.com/paper/2401.15598/) (2024)
- [Fast and Stable Nonconvex Constrained Distributed Optimization: The ELLADA Algorithm](https://mlsanity.com/paper/2004.01977/) (2020)
- [Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows](https://mlsanity.com/paper/2112.01363/) (2021)
- [Finite-Time Convergent Algorithms for Time-Varying Distributed Optimization](https://mlsanity.com/paper/2210.03947/) (2022)
- [Cyber-Social Systems: Modeling, Inference, and Optimal Design](https://mlsanity.com/paper/1903.12371/) (2019)
- [How Clustering Affects the Convergence of Decentralized Optimization over Networks: A Monte-Carlo-based Approach](https://mlsanity.com/paper/2407.01460/) (2024)
- [Parallel and distributed optimization methods for estimation and control in networks](https://mlsanity.com/paper/1302.3103/) (2013)
- [Primal Recovery from Consensus-Based Dual Decomposition for Distributed Convex Optimization](https://mlsanity.com/paper/1503.07678/) (2015)
- [Finite-Time Stability Under Denial of Service](https://mlsanity.com/paper/2005.01040/) (2020)
- [Linear Convergence Rate Analysis of a Class of Exact First-Order Distributed Methods for Weight-Balanced Time-Varying Networks and Uncoordinated Step Sizes](https://mlsanity.com/paper/2007.08837/) (2020)

## Abstract (arXiv)

In recent years, the prevalence of large-scale data-sets and the demand for sophisti-cated learning models have necessitated the development of efficient distributed ma-chine learning (ML) solutions. Convergence speed is a critical factor influencing the practicality and effectiveness of these distributed frameworks. Recently, non-Lipschitz continuous optimization algorithms have been proposed to improve the slow conver-gence rate of the existing linear solutions. The use of signum-based functions is previ-ously considered in consensus and control literature to reach fast convergence in the prescribed time and also to provide robust algorithms to noisy/outlier data. However, as shown in this work, these algorithms lead to an optimality gap and steady-state re-sidual of the objective function in discrete-time setup. This motivates us to investigate the distributed optimization and ML algorithms in terms of trade-off between conver-gence rate and optimality gap. In this direction, we specifically consider the distributed regression problem and check its convergence rate by applying both linear and non-Lipschitz signum-based functions. We check our distributed regression approach by extensive simulations. Our results show that although adopting signum-based func-tions may give faster convergence, it results in large optimality gaps. The findings pre-sented in this paper may contribute to and advance the ongoing discourse of similar distributed algorithms, e.g., for distributed constrained optimization and distributed estimation.