# First-order Newton-type Estimator for Distributed Estimation and Inference

arXiv: https://arxiv.org/abs/1811.11368 · HTML: https://mlsanity.com/paper/1811.11368/

## Key facts

- Cited by 16 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 28 Nov 2018.

## Built on by (most cited first, 16 of 16)

- [Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inference](https://mlsanity.com/paper/2103.02860/) (2021)
- [Statistical Inference of Constrained Stochastic Optimization via Sketched Sequential Quadratic Programming](https://mlsanity.com/paper/2205.13687/) (2022)
- [Communication-Constrained Distributed Quantile Regression with Optimal Statistical Guarantees](https://mlsanity.com/paper/2110.13113/) (2021)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [Distributed Bootstrap for Simultaneous Inference Under High Dimensionality](https://mlsanity.com/paper/2102.10080/) (2021)
- [Adaptive Data Fusion for Multi-task Non-smooth Optimization](https://mlsanity.com/paper/2210.12334/) (2022)
- [Statistical Estimation and Inference via Local SGD in Federated Learning](https://mlsanity.com/paper/2109.01326/) (2021)
- [Federated Learning of Quantile Inference under Local Differential Privacy](https://mlsanity.com/paper/2509.21800/) (2025)
- [Distributed Sparse Regression via Penalization](https://mlsanity.com/paper/2111.06530/) (2021)
- [Byzantine-tolerant distributed learning of finite mixture models](https://mlsanity.com/paper/2407.13980/) (2024)
- [MANDERA: Malicious Node Detection in Federated Learning via Ranking](https://mlsanity.com/paper/2110.11736/) (2021)
- [Distributed Estimation and Inference for Semi-parametric Binary Response Models](https://mlsanity.com/paper/2210.08393/) (2022)
- [A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques](https://mlsanity.com/paper/2403.11163/) (2024)
- [Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize](https://mlsanity.com/paper/2407.10955/) (2024)
- [Minimax and Communication-Efficient Distributed Best Subset Selection with Oracle Property](https://mlsanity.com/paper/2408.17276/) (2024)
- [Refining Covariance Matrix Estimation in Stochastic Gradient Descent Through Bias Reduction](https://mlsanity.com/paper/2604.21203/) (2026)

## Abstract (arXiv)

This paper studies distributed estimation and inference for a general statistical problem with a convex loss that could be non-differentiable. For the purpose of efficient computation, we restrict ourselves to stochastic first-order optimization, which enjoys low per-iteration complexity. To motivate the proposed method, we first investigate the theoretical properties of a straightforward Divide-and-Conquer Stochastic Gradient Descent (DC-SGD) approach. Our theory shows that there is a restriction on the number of machines and this restriction becomes more stringent when the dimension p is large. To overcome this limitation, this paper proposes a new multi-round distributed estimation procedure that approximates the Newton step only using stochastic subgradient. The key component in our method is the proposal of a computationally efficient estimator of Sigma-1 w, where Sigma is the population Hessian matrix and w is any given vector. Instead of estimating Sigma (or Sigma-1) that usually requires the second-order differentiability of the loss, the proposed First-Order Newton-type Estimator (FONE) directly estimates the vector of interest Sigma-1 w as a whole and is applicable to non-differentiable losses. Our estimator also facilitates the inference for the empirical risk minimizer. It turns out that the key term in the limiting covariance has the form of Sigma-1 w, which can be estimated by FONE.