# Communication-Efficient Accurate Statistical Estimation

arXiv: https://arxiv.org/abs/1906.04870 · HTML: https://mlsanity.com/paper/1906.04870/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2019.

## Built on by (most cited first, 30 of 32)

- [Acceleration in Distributed Optimization under Similarity](https://mlsanity.com/paper/2110.12347/) (2021)
- [Distributed Learning of Generalized Linear Causal Networks](https://mlsanity.com/paper/2201.09194/) (2022)
- [Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inference](https://mlsanity.com/paper/2103.02860/) (2021)
- [Federated Data Analytics: A Study on Linear Models](https://mlsanity.com/paper/2206.07786/) (2022)
- [Distributed Saddle-Point Problems Under Similarity](https://mlsanity.com/paper/2107.10706/) (2021)
- [Communication-Constrained Distributed Quantile Regression with Optimal Statistical Guarantees](https://mlsanity.com/paper/2110.13113/) (2021)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [Data splitting improves statistical performance in overparametrized regimes](https://mlsanity.com/paper/2110.10956/) (2021)
- [Distributed Bootstrap for Simultaneous Inference Under High Dimensionality](https://mlsanity.com/paper/2102.10080/) (2021)
- [High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates](https://mlsanity.com/paper/2407.06346/) (2024)
- [Delayed Projection Techniques for Linearly Constrained Problems: Convergence Rates, Acceleration, and Applications](https://mlsanity.com/paper/2101.01505/) (2021)
- [Statistical Estimation and Inference via Local SGD in Federated Learning](https://mlsanity.com/paper/2109.01326/) (2021)
- [Heterogeneous Federated Learning on a Graph](https://mlsanity.com/paper/2209.08737/) (2022)
- [Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recovery](https://mlsanity.com/paper/2405.07552/) (2024)
- [Federated Learning of Quantile Inference under Local Differential Privacy](https://mlsanity.com/paper/2509.21800/) (2025)
- [Distributed Sparse Regression via Penalization](https://mlsanity.com/paper/2111.06530/) (2021)
- [CEDAR: Communication Efficient Distributed Analysis for Regressions](https://mlsanity.com/paper/2207.00306/) (2022)
- [Differentially private sliced inverse regression in the federated paradigm](https://mlsanity.com/paper/2306.06324/) (2023)
- [Distributed Semi-Supervised Sparse Statistical Inference](https://mlsanity.com/paper/2306.10395/) (2023)
- [Byzantine-tolerant distributed learning of finite mixture models](https://mlsanity.com/paper/2407.13980/) (2024)
- [SIM-Shapley: A Stable and Computationally Efficient Approach to Shapley Value Approximation](https://mlsanity.com/paper/2505.08198/) (2025)
- [Divide-and-conquer methods for big data analysis](https://mlsanity.com/paper/2102.10771/) (2021)
- [Communication-efficient Byzantine-robust distributed learning with statistical guarantee](https://mlsanity.com/paper/2103.00373/) (2021)
- [Distributed Estimation and Inference for Semi-parametric Binary Response Models](https://mlsanity.com/paper/2210.08393/) (2022)
- [Local SGD in Overparameterized Linear Regression](https://mlsanity.com/paper/2210.11562/) (2022)
- [Communication-Efficient Distributed Estimation and Inference for Cox's Model](https://mlsanity.com/paper/2302.12111/) (2023)
- [Distributed Linear Regression with Compositional Covariates](https://mlsanity.com/paper/2310.13969/) (2023)
- [A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques](https://mlsanity.com/paper/2403.11163/) (2024)
- [Minimax and Communication-Efficient Distributed Best Subset Selection with Oracle Property](https://mlsanity.com/paper/2408.17276/) (2024)
- [Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine](https://mlsanity.com/paper/2503.07563/) (2025)

## Abstract (arXiv)

When the data are stored in a distributed manner, direct application of traditional statistical inference procedures is often prohibitive due to communication cost and privacy concerns. This paper develops and investigates two Communication-Efficient Accurate Statistical Estimators (CEASE), implemented through iterative algorithms for distributed optimization. In each iteration, node machines carry out computation in parallel and communicate with the central processor, which then broadcasts aggregated information to node machines for new updates. The algorithms adapt to the similarity among loss functions on node machines, and converge rapidly when each node machine has large enough sample size. Moreover, they do not require good initialization and enjoy linear converge guarantees under general conditions. The contraction rate of optimization errors is presented explicitly, with dependence on the local sample size unveiled. In addition, the improved statistical accuracy per iteration is derived. By regarding the proposed method as a multi-step statistical estimator, we show that statistical efficiency can be achieved in finite steps in typical statistical applications. In addition, we give the conditions under which the one-step CEASE estimator is statistically efficient. Extensive numerical experiments on both synthetic and real data validate the theoretical results and demonstrate the superior performance of our algorithms.