mlsanityData to 7 Oct 2026

MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network

arXiv 2406.16633 ↗First posted 24 Jun 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) typically employ an end-to-end (E2E) training paradigm which presents several challenges, including high GPU memory consumption, inefficiency, and difficulties in model parallelization during training. Recent research has sought to address these issues, with one promising approach being local learning. This method involves partitioning the backbone network into gradient-isolated modules and manually designing auxiliary networks to train these local modules. Existing methods often neglect the interaction of information between local modules, leading to myopic issues and a performance gap compared to E2E training. To address these limitations, we propose the Multilaminar Leap Augmented Auxiliary Network (MLAAN). Specifically, MLAAN comprises Multilaminar Local Modules (MLM) and Leap Augmented Modules (LAM). MLM captures both local and global features through independent and cascaded auxiliary networks, alleviating performance issues caused by insufficient global features. However, overly simplistic auxiliary networks can impede MLM's ability to capture global information. To address this, we further design LAM, an enhanced auxiliary network that uses the Exponential Moving Average (EMA) method to facilitate information exchange between local modules, thereby mitigating the shortsightedness resulting from inadequate interaction. The synergy between MLM and LAM has demonstrated excellent performance. Our experiments on the CIFAR-10, STL-10, SVHN, and ImageNet datasets show that MLAAN can be seamlessly integrated into existing local learning frameworks, significantly enhancing their performance and even surpassing end-to-end (E2E) training methods, while also reducing GPU memory consumption.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

31 of 46 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  4. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  5. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  6. 2014Difference Target Propagationcited by 144
  7. 2016Decoupled Neural Interfaces using Synthetic Gradientscited by 135
  8. 2019Training Neural Networks with Local Error Signalscited by 135
  9. 2017Improving Generalization Performance by Switching from Adam to SGDcited by 121
  10. 2018Greedy Layerwise Learning Can Scale to ImageNetcited by 113
  11. 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 100
  12. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 85
19 more
  1. 2019Decoupled Greedy Learning of CNNscited by 76
  2. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  3. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
  4. 2015Towards Biologically Plausible Deep Learningcited by 66
  5. 2021Focus on Local: Detecting Lane Marker from Bottom Up via Key Pointcited by 65
  6. 2022Scaling Forward Gradient With Local Lossescited by 65
  7. 2022Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Passcited by 63
  8. 2022Hebbian Deep Learning Without Feedbackcited by 61
  9. 2019Learned Point Cloud Geometry Compressioncited by 58
  10. 2020LoCo: Local Contrastive Representation Learningcited by 55
  11. 2014Random feedback weights support learning in deep neural networkscited by 47
  12. 2020Local plasticity rules can learn deep representations using self-supervised contrastive predictionscited by 47
  13. 2017Deep supervised learning using local errorscited by 41
  14. 2018Decoupled Parallel Backpropagation with Convergence Guaranteecited by 39
  15. 2018Training Neural Networks Using Features Replaycited by 23
  16. 2021Backpropagation with Biologically Plausible Spatio-Temporal Adjustment For Training Deep Spiking Neural Networkscited by 22
  17. 2023Blockwise Self-Supervised Learning at Scalecited by 22
  18. 2022Fully and Weakly Supervised Referring Expression Segmentation with End-to-End Learningcited by 14
  19. 2024Advancing Supervised Local Learning Beyond Classification with Long-term Feature Bankcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2025MAN++: Scaling Momentum Auxiliary Network for Supervised Local Learning in Vision Taskscited by 3
  2. 2026Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Trainingcited by 3
  3. 2024LAKD-Activation Mapping Distillation Based on Local Learningcited by 1
  4. 2024Replacement Learning: Training Vision Tasks with Fewer Learnable Parameterscited by 1
  5. 2026Replacement Learning: Training Neural Networks with Fewer Parameterscited by 0