mlsanityData to 7 Oct 2026

Zeroth-Order Online Alternating Direction Method of Multipliers: Convergence Analysis and Applications

arXiv 1710.07804 ↗First posted 21 Oct 2017stat.ML cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

In this paper, we design and analyze a new zeroth-order online algorithm, namely, the zeroth-order online alternating direction method of multipliers (ZOO-ADMM), which enjoys dual advantages of being gradient-free operation and employing the ADMM to accommodate complex structured regularizers. Compared to the first-order gradient-based online algorithm, we show that ZOO-ADMM requires √m times more iterations, leading to a convergence rate of O(√m/√T), where m is the number of optimization variables, and T is the number of iterations. To accelerate ZOO-ADMM, we propose two minibatch strategies: gradient sample averaging and observation averaging, resulting in an improved convergence rate of O(√1+q-1m/√T), where q is the minibatch size. In addition to convergence analysis, we also demonstrate ZOO-ADMM to applications in signal processing, statistics, and machine learning.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 2; 2024: 4; 2025: 4; 2026: 4.

22 papers.

Sort:

By area:

  1. 2023DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Trainingcited by 76
  2. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  3. 2024Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inferencecited by 12
  4. 2026Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalizationcited by 11
  5. 2021FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Acceleratorcited by 10
  6. 2021GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsitycited by 10
  7. 2021Statistical Inference for Polyak-Ruppert Averaged Zeroth-order Stochastic Gradient Algorithmcited by 6
  8. 2024Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approachcited by 5
  9. 2025Min-Max Optimisation for Nonconvex-Nonconcave Functions Using a Random Zeroth-Order Extragradient Algorithmcited by 5
  10. 2025ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimizationcited by 4
  11. 2025Sign-SZPO: Provable Preference-based Reinforcement Learning with an Unknown Link Functioncited by 4
  12. 2022Zeroth-Order Hard-Thresholding: Gradient Error vs. Expansivitycited by 3
10 more
  1. 2022Adaptive Zeroth-Order Optimisation of Nonconvex Composite Objectivescited by 1
  2. 2023SODA: Robust Training of Test-Time Data Adaptorscited by 1
  3. 2024End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solvercited by 1
  4. 2024Gradient-Free Method for Heavily Constrained Nonconvex Optimizationcited by 1
  5. 2026Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Samplingcited by 1
  6. 2026ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspacescited by 1
  7. 2021Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examplescited by 0
  8. 2022Adaptive Stochastic Optimisation of Nonconvex Composite Objectivescited by 0
  9. 2025Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clientscited by 0
  10. 2026TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edgecited by 0