# Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss

arXiv: https://arxiv.org/abs/1501.00263 · HTML: https://mlsanity.com/paper/1501.00263/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jan 2015.

## Built on by (most cited first, 7 of 7)

- [How Well Can Transformers Emulate In-context Newton's Method?](https://mlsanity.com/paper/2403.03183/) (2024)
- [Distributed Saddle-Point Problems Under Similarity](https://mlsanity.com/paper/2107.10706/) (2021)
- [High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates](https://mlsanity.com/paper/2407.06346/) (2024)
- [A Privacy-Preserving and Trustable Multi-agent Learning Framework](https://mlsanity.com/paper/2106.01242/) (2021)
- [FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences](https://mlsanity.com/paper/2210.09626/) (2022)
- [Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recovery](https://mlsanity.com/paper/2405.07552/) (2024)
- [Adaptive Consensus: A network pruning approach for decentralized optimization](https://mlsanity.com/paper/2309.02626/) (2023)

## Abstract (arXiv)

We consider distributed convex optimization problems originated from sample average approximation of stochastic optimization, or empirical risk minimization in machine learning. We assume that each machine in the distributed computing system has access to a local empirical loss function, constructed with i.i.d. data sampled from a common distribution. We propose a communication-efficient distributed algorithm to minimize the overall empirical loss, which is the average of the local empirical losses. The algorithm is based on an inexact damped Newton method, where the inexact Newton steps are computed by a distributed preconditioned conjugate gradient method. We analyze its iteration complexity and communication efficiency for minimizing self-concordant empirical loss functions, and discuss the results for distributed ridge regression, logistic regression and binary classification with a smoothed hinge loss. In a standard setting for supervised learning, the required number of communication rounds of the algorithm does not increase with the sample size, and only grows slowly with the number of machines.