# Efficient Convolutional Neural Network Training with Direct Feedback Alignment

arXiv: https://arxiv.org/abs/1901.01986 · HTML: https://mlsanity.com/paper/1901.01986/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2019.

## Built on by (most cited first, 8 of 8)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Low-Variance Forward Gradients using Direct Feedback Alignment and Momentum](https://mlsanity.com/paper/2212.07282/) (2022)
- [Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning](https://mlsanity.com/paper/2401.08603/) (2023)
- [Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment](https://mlsanity.com/paper/2602.15571/) (2026)
- [XConv: Low-memory stochastic backpropagation for convolutional layers](https://mlsanity.com/paper/2106.06998/) (2021)
- [Physical Deep Learning with Biologically Plausible Training Method](https://mlsanity.com/paper/2204.13991/) (2022)
- [A comparative study of back propagation and its alternatives on multilayer perceptrons](https://mlsanity.com/paper/2206.06098/) (2022)
- [Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment](https://mlsanity.com/paper/2605.30638/) (2026)

## Abstract (arXiv)

There were many algorithms to substitute the back-propagation (BP) in the deep neural network (DNN) training. However, they could not become popular because their training accuracy and the computational efficiency were worse than BP. One of them was direct feedback alignment (DFA), but it showed low training performance especially for the convolutional neural network (CNN). In this paper, we overcome the limitation of the DFA algorithm by combining with the conventional BP during the CNN training. To improve the training stability, we also suggest the feedback weight initialization method by analyzing the patterns of the fixed random matrices in the DFA. Finally, we propose the new training algorithm, binary direct feedback alignment (BDFA) to minimize the computational cost while maintaining the training accuracy compared with the DFA. In our experiments, we use the CIFAR-10 and CIFAR-100 dataset to simulate the CNN learning from the scratch and apply the BDFA to the online learning based object tracking application to examine the training in the small dataset environment. Our proposed algorithms show better performance than conventional BP in both two different training tasks especially when the dataset is small.