# Integration of Leaky-Integrate-and-Fire-Neurons in Deep Learning Architectures

arXiv: https://arxiv.org/abs/2004.13532 · HTML: https://mlsanity.com/paper/2004.13532/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2020.

## Built on by (most cited first, 7 of 7)

- [Neural Network based Successor Representations of Space and Language](https://mlsanity.com/paper/2202.11190/) (2022)
- [Word class representations spontaneously emerge in a deep neural network trained on next word prediction](https://mlsanity.com/paper/2302.07588/) (2023)
- [Neural Network based Formation of Cognitive Maps of Semantic Spaces and the Emergence of Abstract Concepts](https://mlsanity.com/paper/2210.16062/) (2022)
- [Flashy Backdoor: Real-world Environment Backdoor Attack on SNNs with DVS Cameras](https://mlsanity.com/paper/2411.03022/) (2024)
- [Analyzing Narrative Processing in Large Language Models (LLMs): Using GPT4 to test BERT](https://mlsanity.com/paper/2405.02024/) (2024)
- [Predictive coding and stochastic resonance as fundamental principles of auditory perception](https://mlsanity.com/paper/2204.03354/) (2022)
- [Evaluation of Bio-Inspired Models under Different Learning Settings For Energy Efficiency in Network Traffic Prediction](https://mlsanity.com/paper/2412.17565/) (2024)

## Abstract (arXiv)

Up to now, modern Machine Learning is mainly based on fitting high dimensional functions to enormous data sets, taking advantage of huge hardware resources. We show that biologically inspired neuron models such as the Leaky-Integrate-and-Fire (LIF) neurons provide novel and efficient ways of information encoding. They can be integrated in Machine Learning models, and are a potential target to improve Machine Learning performance. Thus, we derived simple update-rules for the LIF units from the differential equations, which are easy to numerically integrate. We apply a novel approach to train the LIF units supervisedly via backpropagation, by assigning a constant value to the derivative of the neuron activation function exclusively for the backpropagation step. This simple mathematical trick helps to distribute the error between the neurons of the pre-connected layer. We apply our method to the IRIS blossoms image data set and show that the training technique can be used to train LIF neurons on image classification tasks. Furthermore, we show how to integrate our method in the KERAS (tensorflow) framework and efficiently run it on GPUs. To generate a deeper understanding of the mechanisms during training we developed interactive illustrations, which we provide online. With this study we want to contribute to the current efforts to enhance Machine Intelligence by integrating principles from biology.