# Distributed Distributionally Robust Optimization with Non-Convex Objectives

arXiv: https://arxiv.org/abs/2210.07588 · HTML: https://mlsanity.com/paper/2210.07588/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 66 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 29 of 29)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems](https://mlsanity.com/paper/1906.00331/) (2019)
- [A backdoor attack against LSTM-based text classification systems](https://mlsanity.com/paper/1905.12457/) (2019)
- [Distributionally Robust Stochastic Optimization with Wasserstein Distance](https://mlsanity.com/paper/1604.02199/) (2016)
- [Large-Scale Methods for Distributionally Robust Optimization](https://mlsanity.com/paper/2010.05893/) (2020)
- [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://mlsanity.com/paper/1910.06378/) (2019)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Distributionally Robust Federated Averaging](https://mlsanity.com/paper/2102.12660/) (2021)
- [Federated Reconstruction: Partially Local Federated Learning](https://mlsanity.com/paper/2102.03448/) (2021)
- [Efficient Algorithms for Smooth Minimax Optimization](https://mlsanity.com/paper/1907.01543/) (2019)
- [A Unified Single-loop Alternating Gradient Projection Algorithm for Nonconvex-Concave and Convex-Nonconcave Minimax Problems](https://mlsanity.com/paper/2006.02032/) (2020)
- [Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications](https://mlsanity.com/paper/1902.08294/) (2019)
- [On the Fenchel Duality between Strong Convexity and Lipschitz Continuous Gradient](https://mlsanity.com/paper/1803.06573/) (2018)
- [An Online Method for A Class of Distributionally Robust Optimization with Non-Convex Objectives](https://mlsanity.com/paper/2006.10138/) (2020)
- [Robust Optimization over Multiple Domains](https://mlsanity.com/paper/1805.07588/) (2018)
- [Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients](https://mlsanity.com/paper/1909.07588/) (2019)
- [Asynchronous Stochastic Optimization Robust to Arbitrary Delays](https://mlsanity.com/paper/2106.11879/) (2021)
- [Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning](https://mlsanity.com/paper/2102.00655/) (2021)
- [A Practical Guide to Robust Optimization](https://mlsanity.com/paper/1501.02634/) (2015)
- [Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis](https://mlsanity.com/paper/1509.02597/) (2015)
- [Learning Distributionally Robust Models at Scale via Composite Optimization](https://mlsanity.com/paper/2203.09607/) (2022)
- [A cutting surface algorithm for semi-infinite convex programming with an application to moment robust optimization](https://mlsanity.com/paper/1306.3437/) (2013)
- [Robustness analytics to data heterogeneity in edge computing](https://mlsanity.com/paper/2002.05038/) (2020)

## Built on by (most cited first, 14 of 14)

- [Asynchronous Distributed Bilevel Optimization](https://mlsanity.com/paper/2212.10048/) (2022)
- [Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization](https://mlsanity.com/paper/2410.07018/) (2024)
- [Provably Convergent Federated Trilevel Learning](https://mlsanity.com/paper/2312.11835/) (2023)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Federated Distributionally Robust Optimization with Non-Convex Objectives: Algorithm and Analysis](https://mlsanity.com/paper/2307.14364/) (2023)
- [Cellular Traffic Prediction via Byzantine-robust Asynchronous Federated Learning](https://mlsanity.com/paper/2505.19263/) (2025)
- [Argus: Federated Non-convex Bilevel Learning over 6G Space-Air-Ground Integrated Network](https://mlsanity.com/paper/2505.09106/) (2025)
- [CPMLHO:Hyperparameter Tuning via Cutting Plane and Mixed-Level Optimization](https://mlsanity.com/paper/2212.06150/) (2022)
- [Multi-Level Association Rule Mining for Wireless Network Time Series Data](https://mlsanity.com/paper/2212.07860/) (2022)
- [Generalizing Few Data to Unseen Domains Flexibly Based on Label Smoothing Integrated with Distributionally Robust Optimization](https://mlsanity.com/paper/2408.05082/) (2024)
- [Synergistic Localization and Sensing in MIMO-OFDM Systems via Mixed-Integer Bilevel Learning](https://mlsanity.com/paper/2507.07118/) (2025)
- [An Adaptive Differentially Private Federated Learning Framework](https://mlsanity.com/paper/2602.06838/) (2026)
- [AsynDBT: Asynchronous Distributed Bilevel Tuning for efficient In-Context Learning with Large Language Models](https://mlsanity.com/paper/2602.17694/) (2026)
- [First-order Constrained Trilevel Optimization Over Distributed Networks for Robust Coreset Selection](https://mlsanity.com/paper/2607.27632/) (2026)

## Abstract (arXiv)

Distributionally Robust Optimization (DRO), which aims to find an optimal decision that minimizes the worst case cost over the ambiguity set of probability distribution, has been widely applied in diverse applications, e.g., network behavior analysis, risk management, etc. However, existing DRO techniques face three key challenges: 1) how to deal with the asynchronous updating in a distributed environment; 2) how to leverage the prior distribution effectively; 3) how to properly adjust the degree of robustness according to different scenarios. To this end, we propose an asynchronous distributed algorithm, named Asynchronous Single-looP alternatIve gRadient projEction (ASPIRE) algorithm with the itErative Active SEt method (EASE) to tackle the distributed distributionally robust optimization (DDRO) problem. Furthermore, a new uncertainty set, i.e., constrained D-norm uncertainty set, is developed to effectively leverage the prior distribution and flexibly control the degree of robustness. Finally, our theoretical analysis elucidates that the proposed algorithm is guaranteed to converge and the iteration complexity is also analyzed. Extensive empirical studies on real-world datasets demonstrate that the proposed method can not only achieve fast convergence, and remain robust against data heterogeneity as well as malicious attacks, but also tradeoff robustness with performance.