# Deep supervised learning using local errors

arXiv: https://arxiv.org/abs/1711.06756 · HTML: https://mlsanity.com/paper/1711.06756/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2017.

## Built on by (most cited first, 30 of 41)

- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Revisiting Locally Supervised Learning: an Alternative to End-to-end Training](https://mlsanity.com/paper/2101.10832/) (2021)
- [Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence](https://mlsanity.com/paper/2106.01288/) (2021)
- [Towards Biologically Plausible Convolutional Networks](https://mlsanity.com/paper/2106.13031/) (2021)
- [OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses](https://mlsanity.com/paper/2204.02426/) (2022)
- [Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits](https://mlsanity.com/paper/2303.18187/) (2023)
- [Momentum Auxiliary Network for Supervised Local Learning](https://mlsanity.com/paper/2407.05623/) (2024)
- [Scaling Supervised Local Learning with Augmented Auxiliary Networks](https://mlsanity.com/paper/2402.17318/) (2024)
- [End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training](https://mlsanity.com/paper/2402.09050/) (2024)
- [Decoupled Greedy Learning of CNNs for Synchronous and Asynchronous Distributed Learning](https://mlsanity.com/paper/2106.06401/) (2021)
- [Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods](https://mlsanity.com/paper/2101.03419/) (2021)
- [FTBC: Forward Temporal Bias Correction for Optimizing ANN-SNN Conversion](https://mlsanity.com/paper/2403.18388/) (2024)
- [MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network](https://mlsanity.com/paper/2406.16633/) (2024)
- [HPFF: Hierarchical Locally Supervised Learning with Patch Feature Fusion](https://mlsanity.com/paper/2407.05638/) (2024)
- [Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation](https://mlsanity.com/paper/2406.05222/) (2024)
- [Efficient Training of Spiking Neural Networks with Temporally-Truncated Local Backpropagation through Time](https://mlsanity.com/paper/2201.07210/) (2021)
- [Feed-Forward Optimization With Delayed Feedback for Neural Network Training](https://mlsanity.com/paper/2304.13372/) (2023)
- [MAN++: Scaling Momentum Auxiliary Network for Supervised Local Learning in Vision Tasks](https://mlsanity.com/paper/2507.16279/) (2025)
- [Layer-Parallel Training of Residual Networks with Auxiliary-Variable Networks](https://mlsanity.com/paper/2112.05387/) (2021)
- [Seeking Next Layer Neurons' Attention for Error-Backpropagation-Like Training in a Multi-Agent Network Framework](https://mlsanity.com/paper/2310.09952/) (2023)
- [Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment](https://mlsanity.com/paper/2602.15571/) (2026)
- [Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers](https://mlsanity.com/paper/2102.04697/) (2021)
- [BackLink: Supervised Local Training with Backward Links](https://mlsanity.com/paper/2205.07141/) (2022)
- [Learning with Local Gradients at the Edge](https://mlsanity.com/paper/2208.08503/) (2022)
- [Block-wise Training of Residual Networks via the Minimizing Movement Scheme](https://mlsanity.com/paper/2210.00949/) (2022)
- [QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architectures](https://mlsanity.com/paper/2212.12866/) (2022)
- [Leader-Follower Neural Networks with Local Error Signals Inspired by Complex Collectives](https://mlsanity.com/paper/2310.07885/) (2023)
- [Forward Learning of Graph Neural Networks](https://mlsanity.com/paper/2403.11004/) (2024)
- [Replacement Learning: Training Vision Tasks with Fewer Learnable Parameters](https://mlsanity.com/paper/2410.01239/) (2024)
- [Training Multi-Layer Binary Neural Networks With Local Binary Error Signals](https://mlsanity.com/paper/2412.00119/) (2024)

## Abstract (arXiv)

Error backpropagation is a highly effective mechanism for learning high-quality hierarchical features in deep networks. Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers. Learning using delayed and non-local errors makes it hard to reconcile backpropagation with the learning mechanisms observed in biological neural networks as it requires the neurons to maintain a memory of the input long enough until the higher-layer errors arrive. In this paper, we propose an alternative learning mechanism where errors are generated locally in each layer using fixed, random auxiliary classifiers. Lower layers could thus be trained independently of higher layers and training could either proceed layer by layer, or simultaneously in all layers using local error information. We address biological plausibility concerns such as weight symmetry requirements and show that the proposed learning mechanism based on fixed, broad, and random tuning of each neuron to the classification categories outperforms the biologically-motivated feedback alignment learning technique on the MNIST, CIFAR10, and SVHN datasets, approaching the performance of standard backpropagation. Our approach highlights a potential biological mechanism for the supervised, or task-dependent, learning of feature hierarchies. In addition, we show that it is well suited for learning deep networks in custom hardware where it can drastically reduce memory traffic and data communication overheads.