# Improving Neural Network Learning Through Dual Variable Learning Rates

arXiv: https://arxiv.org/abs/2002.03428 · HTML: https://mlsanity.com/paper/2002.03428/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 3 of 3)

- [Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond](https://mlsanity.com/paper/2403.03562/) (2024)
- [Evolving choice hysteresis in reinforcement learning: comparing the adaptive value of positivity bias and gradual perseveration](https://mlsanity.com/paper/2410.19434/) (2024)
- [We Don't Need No Adam, All We Need Is EVE: On The Variance of Dual Learning Rate And Beyond](https://mlsanity.com/paper/2308.10740/) (2023)

## Abstract (arXiv)

This paper introduces and evaluates a novel training method for neural networks: Dual Variable Learning Rates (DVLR). Building on insights from behavioral psychology, the dual learning rates are used to emphasize correct and incorrect responses differently, thereby making the feedback to the network more specific. Further, the learning rates are varied as a function of the network's performance, thereby making it more efficient. DVLR was implemented on three types of networks: feedforward, convolutional, and residual, and two domains: MNIST and CIFAR-10. The results suggest a consistently improved accuracy, demonstrating that DVLR is a promising, psychologically motivated technique for training neural network models.