LightFF: Lightweight Inference for Forward-Forward Algorithm
Not on the map. Why?
Abstract (arXiv)
The human brain performs tasks with an outstanding energy efficiency, i.e., with approximately 20 Watts. The state-of-the-art Artificial/Deep Neural Networks (ANN/DNN), on the other hand, have recently been shown to consume massive amounts of energy. The training of these ANNs/DNNs is done almost exclusively based on the back-propagation algorithm, which is known to be biologically implausible. This has led to a new generation of forward-only techniques, including the Forward-Forward algorithm. In this paper, we propose a lightweight inference scheme specifically designed for DNNs trained using the Forward-Forward algorithm. We have evaluated our proposed lightweight inference scheme in the case of the MNIST and CIFAR datasets, as well as two real-world applications, namely, epileptic seizure detection and cardiac arrhythmia classification using wearable technologies, where complexity overheads/energy consumption is a major constraint, and demonstrate its relevance. Our code is available at https://github.com/AminAminifar/LightFF.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 30 of its 58 references that are in the map. First posted on arXiv 8 Apr 2024.
Builds on
30 of 58 references.
References · 30
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
- 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
- 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 251
- 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
- 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
- 2017Adaptive Neural Networks for Efficient Inferencecited by 161
- 2016Decoupled Neural Interfaces using Synthetic Gradientscited by 135
- 2019Training Neural Networks with Local Error Signalscited by 135
- 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 114
- 2018Dendritic cortical microcircuits approximate the backpropagation algorithmcited by 96
18 more
- 2020Predictive Coding Approximates Backprop along Arbitrary Computation Graphscited by 89
- 2021ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reductioncited by 84
- 2018ECG Heartbeat Classification: A Deep Transferable Representationcited by 76
- 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
- 2016Towards deep learning with segregated dendritescited by 71
- 2020Why should we add early exits to neural networks?cited by 70
- 2015Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognitioncited by 67
- 2022Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Passcited by 63
- 2022Hebbian Deep Learning Without Feedbackcited by 61
- 2015How Important is Weight Symmetry in Backpropagation?cited by 57
- 2020Depth Uncertainty in Neural Networkscited by 56
- 2017Understanding Synthetic Gradients and Decoupled Neural Interfacescited by 35
- 2023The Predictive Forward-Forward Algorithmcited by 35
- 2021Credit Assignment in Neural Networks through Deep Feedback Controlcited by 28
- 2023The Cascaded Forward Algorithm for Neural Network Trainingcited by 17
- 2022An Energy and Carbon Footprint Analysis of Distributed and Federated Learningcited by 16
- 2023Forward Learning with Top-Down Feedback: Empirical and Analytical Characterizationcited by 13
- 2023Representation Learning via Manifold Flattening and Reconstructioncited by 11
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.