mlsanityData to 14 Oct 2026

Convergence of alternating minimisation algorithms for dictionary learning

arXiv 2304.01768 ↗First posted 4 Apr 2023math.OC cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

In this paper we derive sufficient conditions for the convergence of two popular alternating minimisation algorithms for dictionary learning - the Method of Optimal Directions (MOD) and Online Dictionary Learning (ODL), which can also be thought of as approximative K-SVD. We show that given a well-behaved initialisation that is either within distance at most 1/log(K) to the generating dictionary or has a special structure ensuring that each element of the initialisation only points to one generating element, both algorithms will converge with geometric convergence rate to the generating dictionary. This is done even for data models with non-uniform distributions on the supports of the sparse coefficients. These allow the appearance frequency of the dictionary elements to vary heavily and thus model real data more closely.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 42 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

14 of 42 references.

References · 14

  1. 2015Simple, Efficient, and Neural Algorithms for Sparse Codingcited by 37
  2. 2013Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimizationcited by 24
  3. 2015Complete Dictionary Recovery over the Sphere II: Recovery by Riemannian Trust-region Methodcited by 24
  4. 2013New Algorithms for Learning Incoherent and Overcomplete Dictionariescited by 22
  5. 2014Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Methodcited by 22
  6. 2015Complete Dictionary Recovery over the Spherecited by 19
  7. 2014On the Computational Intractability of Exact and Approximate Dictionary Learningcited by 14
  8. 2014Sparse and spurious: dictionary learning with noise and outlierscited by 11
  9. 2013A Clustering Approach to Learn Sparsely-Used Overcomplete Dictionariescited by 7
  10. 2014Local Identification of Overcomplete Dictionariescited by 7
  11. 2015Convergence radius and sample complexity of ITKM algorithms for dictionary learningcited by 4
  12. 2017Alternating minimization for dictionary learning: Local Convergence Guaranteescited by 4
2 more
  1. 2018Dictionary learning -- from local towards global and adaptivecited by 2
  2. 2022Adapted variable density subsampling for compressed sensingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Coupled Training with Privileged Information and Unlabeled Datacited by 1