mlsanityData to 7 Oct 2026

A Proximal Block Coordinate Descent Algorithm for Deep Neural Network Training

arXiv 1803.09082 ↗First posted 24 Mar 2018stat.ML cs.LG math.OCcited by 15

Not on the map. Why?

Abstract (arXiv)

Training deep neural networks (DNNs) efficiently is a challenge due to the associated highly nonconvex optimization. The backpropagation (backprop) algorithm has long been the most widely used algorithm for gradient computation of parameters of DNNs and is used along with gradient descent-type algorithms for this optimization task. Recent work have shown the efficiency of block coordinate descent (BCD) type methods empirically for training DNNs. In view of this, we propose a novel algorithm based on the BCD method for training DNNs and provide its global convergence results built upon the powerful framework of the Kurdyka-Lojasiewicz (KL) property. Numerical experiments on standard datasets demonstrate its competitive efficiency against standard optimizers with backprop.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 5; 2023: 1; 2024: 2; 2025: 3.

15 papers.

Sort:

  1. 2022Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscapecited by 39
  2. 2022A Theoretical Framework for Inference Learningcited by 17
  3. 2021LocoProp: Enhancing BackProp via Local Loss Optimizationcited by 15
  4. 20220/1 Deep Neural Networks via Block Coordinate Descentcited by 10
  5. 2024BlockLLM: Memory-Efficient Adaptation of LLMs by Selecting and Optimizing the Right Coordinate Blockscited by 9
  6. 2021Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methodscited by 5
  7. 2022An Inexact Augmented Lagrangian Algorithm for Training Leaky ReLU Neural Network with Group Sparsitycited by 5
  8. 2022Convergence Rates of Training Deep Neural Networks via Alternating Minimization Methodscited by 4
  9. 2021Personalized On-Device E-health Analytics with Decentralized Block Coordinate Descentcited by 2
  10. 2023On Model Compression for Neural Networks: Framework, Algorithm, and Convergence Guaranteecited by 1
  11. 2024Enhancing Convergence of Decentralized Gradient Tracking under the KL Propertycited by 1
  12. 2025A Triple-Inertial Accelerated Alternating Optimization Method for Deep Learning Trainingcited by 1
3 more
  1. 2021Learning DNN networks using un-rectifying ReLU with compressed sensing applicationcited by 0
  2. 2025Exploiting Block Coordinate Descent for Cost-Effective LLM Model Trainingcited by 0
  3. 2025Block Coordinate Descent for Neural Networks Provably Finds Global Minimacited by 0