Energy-Efficient Visual Search by Eye Movement and Low-Latency Spiking Neural Network
Not on the map. Why?
Abstract (arXiv)
Human vision incorporates non-uniform resolution retina, efficient eye movement strategy, and spiking neural network (SNN) to balance the requirements in visual field size, visual resolution, energy cost, and inference latency. These properties have inspired interest in developing human-like computer vision. However, existing models haven't fully incorporated the three features of human vision, and their learned eye movement strategies haven't been compared with human's strategy, making the models' behavior difficult to interpret. Here, we carry out experiments to examine human visual search behaviors and establish the first SNN-based visual search model. The model combines an artificial retina with spiking feature extraction, memory, and saccade decision modules, and it employs population coding for fast and efficient saccade decisions. The model can learn either a human-like or a near-optimal fixation strategy, outperform humans in search speed and accuracy, and achieve high energy efficiency through short saccade decision latency and sparse activation. It also suggests that the human search strategy is suboptimal in terms of search speed. Our work connects modeling of vision in neuroscience and machine learning and sheds light on developing more energy-efficient computer vision algorithms.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 24 of its 100 references that are in the map. First posted on arXiv 10 Oct 2023.
Builds on
24 of 100 references.
References · 24
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,185
- 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 403
- 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
- 2014Recurrent Models of Visual Attentioncited by 344
- 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 232
- 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 176
- 2023Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networkscited by 120
- 2020Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Controlcited by 44
- 2020Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modulescited by 38
- 2019Saccader: Improving Accuracy of Hard Attention Models for Visioncited by 35
- 2021A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detectioncited by 31
12 more
- 2021Backpropagation with Biologically Plausible Spatio-Temporal Adjustment For Training Deep Spiking Neural Networkscited by 23
- 2021Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Visioncited by 22
- 2019Low-Power Neuromorphic Hardware for Signal Processing Applicationscited by 21
- 2020Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networkscited by 20
- 2021One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latencycited by 17
- 2022Deep Reinforcement Learning with Spiking Q-learningcited by 16
- 2014Object Detection Through Exploration With A Foveated Visual Fieldcited by 14
- 2021A Probabilistic Hard Attention Model For Sequentially Observed Scenescited by 12
- 2022Consistency driven Sequential Transformers Attention Model for Partially Observable Scenescited by 11
- 2021RetinaNet Object Detector based on Analog-to-Spiking Neural Network Conversioncited by 3
- 2017Learning Fixation Point Strategy for Object Detection and Classificationcited by 1
- 2022GliTr: Glimpse Transformers with Spatiotemporal Consistency for Online Action Predictioncited by 1
Built on by
Not cited yet.