# MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network

arXiv: https://arxiv.org/abs/2406.16633 · HTML: https://mlsanity.com/paper/2406.16633/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Decoupled Neural Interfaces using Synthetic Gradients](https://mlsanity.com/paper/1608.05343/) (2016)
- [Training Neural Networks with Local Error Signals](https://mlsanity.com/paper/1901.06656/) (2019)
- [Improving Generalization Performance by Switching from Adam to SGD](https://mlsanity.com/paper/1712.07628/) (2017)
- [Greedy Layerwise Learning Can Scale to ImageNet](https://mlsanity.com/paper/1812.11446/) (2018)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [Putting An End to End-to-End: Gradient-Isolated Learning of Representations](https://mlsanity.com/paper/1905.11786/) (2019)
- [Decoupled Greedy Learning of CNNs](https://mlsanity.com/paper/1901.08164/) (2019)
- [Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction](https://mlsanity.com/paper/2103.04174/) (2021)
- [Revisiting Locally Supervised Learning: an Alternative to End-to-end Training](https://mlsanity.com/paper/2101.10832/) (2021)
- [Towards Biologically Plausible Deep Learning](https://mlsanity.com/paper/1502.04156/) (2015)
- [Focus on Local: Detecting Lane Marker from Bottom Up via Key Point](https://mlsanity.com/paper/2105.13680/) (2021)
- [Scaling Forward Gradient With Local Losses](https://mlsanity.com/paper/2210.03310/) (2022)
- [Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass](https://mlsanity.com/paper/2201.11665/) (2022)
- [Hebbian Deep Learning Without Feedback](https://mlsanity.com/paper/2209.11883/) (2022)
- [Learned Point Cloud Geometry Compression](https://mlsanity.com/paper/1909.12037/) (2019)
- [LoCo: Local Contrastive Representation Learning](https://mlsanity.com/paper/2008.01342/) (2020)
- [Random feedback weights support learning in deep neural networks](https://mlsanity.com/paper/1411.0247/) (2014)
- [Local plasticity rules can learn deep representations using self-supervised contrastive predictions](https://mlsanity.com/paper/2010.08262/) (2020)
- [Deep supervised learning using local errors](https://mlsanity.com/paper/1711.06756/) (2017)
- [Decoupled Parallel Backpropagation with Convergence Guarantee](https://mlsanity.com/paper/1804.10574/) (2018)
- [Training Neural Networks Using Features Replay](https://mlsanity.com/paper/1807.04511/) (2018)
- [Backpropagation with Biologically Plausible Spatio-Temporal Adjustment For Training Deep Spiking Neural Networks](https://mlsanity.com/paper/2110.08858/) (2021)
- [Blockwise Self-Supervised Learning at Scale](https://mlsanity.com/paper/2302.01647/) (2023)
- [Fully and Weakly Supervised Referring Expression Segmentation with End-to-End Learning](https://mlsanity.com/paper/2212.10278/) (2022)

## Built on by (most cited first, 5 of 5)

- [Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training](https://mlsanity.com/paper/2605.04913/) (2026)
- [MAN++: Scaling Momentum Auxiliary Network for Supervised Local Learning in Vision Tasks](https://mlsanity.com/paper/2507.16279/) (2025)
- [LAKD-Activation Mapping Distillation Based on Local Learning](https://mlsanity.com/paper/2408.11478/) (2024)
- [Replacement Learning: Training Vision Tasks with Fewer Learnable Parameters](https://mlsanity.com/paper/2410.01239/) (2024)
- [Replacement Learning: Training Neural Networks with Fewer Parameters](https://mlsanity.com/paper/2605.19533/) (2026)

## Abstract (arXiv)

Deep neural networks (DNNs) typically employ an end-to-end (E2E) training paradigm which presents several challenges, including high GPU memory consumption, inefficiency, and difficulties in model parallelization during training. Recent research has sought to address these issues, with one promising approach being local learning. This method involves partitioning the backbone network into gradient-isolated modules and manually designing auxiliary networks to train these local modules. Existing methods often neglect the interaction of information between local modules, leading to myopic issues and a performance gap compared to E2E training. To address these limitations, we propose the Multilaminar Leap Augmented Auxiliary Network (MLAAN). Specifically, MLAAN comprises Multilaminar Local Modules (MLM) and Leap Augmented Modules (LAM). MLM captures both local and global features through independent and cascaded auxiliary networks, alleviating performance issues caused by insufficient global features. However, overly simplistic auxiliary networks can impede MLM's ability to capture global information. To address this, we further design LAM, an enhanced auxiliary network that uses the Exponential Moving Average (EMA) method to facilitate information exchange between local modules, thereby mitigating the shortsightedness resulting from inadequate interaction. The synergy between MLM and LAM has demonstrated excellent performance. Our experiments on the CIFAR-10, STL-10, SVHN, and ImageNet datasets show that MLAAN can be seamlessly integrated into existing local learning frameworks, significantly enhancing their performance and even surpassing end-to-end (E2E) training methods, while also reducing GPU memory consumption.