# Mirror descent of Hopfield model

arXiv: https://arxiv.org/abs/2211.15880 · HTML: https://mlsanity.com/paper/2211.15880/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Policy Mirror Descent for Reinforcement Learning: Linear Convergence, New Sampling Complexity, and Generalized Problem Classes](https://mlsanity.com/paper/2102.00135/) (2021)
- [Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence](https://mlsanity.com/paper/2105.11066/) (2021)
- [PCANet: A Simple Deep Learning Baseline for Image Classification?](https://mlsanity.com/paper/1404.3606/) (2014)
- [The Information Geometry of Mirror Descent](https://mlsanity.com/paper/1310.7780/) (2013)
- [Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization](https://mlsanity.com/paper/1906.03830/) (2019)
- [PCA-Initialized Deep Neural Networks Applied To Document Image Analysis](https://mlsanity.com/paper/1702.00177/) (2017)
- [Data-driven Weight Initialization with Sylvester Solvers](https://mlsanity.com/paper/2105.10335/) (2021)
- [Linear discriminant initialization for feed-forward neural networks](https://mlsanity.com/paper/2007.12782/) (2020)

## Built on by (most cited first, 2 of 2)

- [Multiplicative learning from observation-prediction ratios](https://mlsanity.com/paper/2503.10144/) (2025)
- [Mirror Descent on Riemannian Manifolds](https://mlsanity.com/paper/2603.17527/) (2026)

## Abstract (arXiv)

Mirror descent is an elegant optimization technique that leverages a dual space of parametric models to perform gradient descent. While originally developed for convex optimization, it has increasingly been applied in the field of machine learning. In this study, we propose a novel approach for utilizing mirror descent to initialize the parameters of neural networks. Specifically, we demonstrate that by using the Hopfield model as a prototype for neural networks, mirror descent can effectively train the model with significantly improved performance compared to traditional gradient descent methods that rely on random parameter initialization. Our findings highlight the potential of mirror descent as a promising initialization technique for enhancing the optimization of machine learning models.