# Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades

arXiv: https://arxiv.org/abs/1507.07629 · HTML: https://mlsanity.com/paper/1507.07629/

## Key facts

- Cited by 314 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2015.

## Built on by (most cited first, 30 of 314)

- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras](https://mlsanity.com/paper/2112.01041/) (2021)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Time-Ordered Recent Event (TORE) Volumes for Event Cameras](https://mlsanity.com/paper/2103.06108/) (2021)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Neuromorphic Data Augmentation for Training Spiking Neural Networks](https://mlsanity.com/paper/2203.06145/) (2022)
- [Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline](https://mlsanity.com/paper/2304.06793/) (2023)
- [Graph-based Asynchronous Event Processing for Rapid Object Recognition](https://mlsanity.com/paper/2308.14419/) (2023)
- [Event Transformer. A sparse-aware solution for efficient event data processing](https://mlsanity.com/paper/2204.03355/) (2022)
- [Event Camera Data Pre-training](https://mlsanity.com/paper/2301.01928/) (2023)
- [UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All](https://mlsanity.com/paper/2403.12532/) (2024)
- [EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation](https://mlsanity.com/paper/2111.12341/) (2021)
- [HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors](https://mlsanity.com/paper/2211.09648/) (2022)
- [Sparse Spiking Gradient Descent](https://mlsanity.com/paper/2105.08810/) (2021)
- [Masked Event Modeling: Self-Supervised Pretraining for Event Cameras](https://mlsanity.com/paper/2212.10368/) (2022)
- [Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State](https://mlsanity.com/paper/2109.14247/) (2021)
- [A Voxel Graph CNN for Object Classification with Event Cameras](https://mlsanity.com/paper/2106.00216/) (2021)
- [Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.02618/) (2021)
- [EventCLIP: Adapting CLIP for Event-based Object Recognition](https://mlsanity.com/paper/2306.06354/) (2023)
- [TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks](https://mlsanity.com/paper/2206.10177/) (2022)
- [EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding](https://mlsanity.com/paper/2308.03135/) (2023)
- [Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translation](https://mlsanity.com/paper/2109.01801/) (2021)
- [Pushing the Limits of Asynchronous Graph-based Object Detection with Event Cameras](https://mlsanity.com/paper/2211.12324/) (2022)
- [SpikeMS: Deep Spiking Neural Network for Motion Segmentation](https://mlsanity.com/paper/2105.06562/) (2021)
- [EventDrop: data augmentation for event-based learning](https://mlsanity.com/paper/2106.05836/) (2021)
- [Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow](https://mlsanity.com/paper/2112.06242/) (2021)
- [Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events](https://mlsanity.com/paper/2308.09383/) (2023)

## Abstract (arXiv)

Creating datasets for Neuromorphic Vision is a challenging task. A lack of available recordings from Neuromorphic Vision sensors means that data must typically be recorded specifically for dataset creation rather than collecting and labelling existing data. The task is further complicated by a desire to simultaneously provide traditional frame-based recordings to allow for direct comparison with traditional Computer Vision algorithms. Here we propose a method for converting existing Computer Vision static image datasets into Neuromorphic Vision datasets using an actuated pan-tilt camera platform. Moving the sensor rather than the scene or image is a more biologically realistic approach to sensing and eliminates timing artifacts introduced by monitor updates when simulating motion on a computer monitor. We present conversion of two popular image datasets (MNIST and Caltech101) which have played important roles in the development of Computer Vision, and we provide performance metrics on these datasets using spike-based recognition algorithms. This work contributes datasets for future use in the field, as well as results from spike-based algorithms against which future works can compare. Furthermore, by converting datasets already popular in Computer Vision, we enable more direct comparison with frame-based approaches.