# Temporal Reversal Regularization for Spiking Neural Networks: Hybrid Spatio-Temporal Invariance for Generalization

arXiv: https://arxiv.org/abs/2408.09108 · HTML: https://mlsanity.com/paper/2408.09108/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 17 Aug 2024.

## Builds on (most cited first, 30 of 32)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [Spike-driven Transformer](https://mlsanity.com/paper/2307.01694/) (2023)
- [Advancing Spiking Neural Networks towards Deep Residual Learning](https://mlsanity.com/paper/2112.08954/) (2021)
- [Deep Directly-Trained Spiking Neural Networks for Object Detection](https://mlsanity.com/paper/2307.11411/) (2023)
- [Rewrite the Stars](https://mlsanity.com/paper/2403.19967/) (2024)
- [Neuromorphic Data Augmentation for Training Spiking Neural Networks](https://mlsanity.com/paper/2203.06145/) (2022)
- [Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline](https://mlsanity.com/paper/2304.06793/) (2023)
- [QKFormer: Hierarchical Spiking Transformer using Q-K Attention](https://mlsanity.com/paper/2403.16552/) (2024)
- [SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentiation](https://mlsanity.com/paper/2308.10873/) (2023)
- [Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks](https://mlsanity.com/paper/2007.01204/) (2020)
- [Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikes](https://mlsanity.com/paper/2302.10685/) (2023)
- [On the Importance of Asymmetry for Siamese Representation Learning](https://mlsanity.com/paper/2204.00613/) (2022)
- [Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks](https://mlsanity.com/paper/2312.06372/) (2023)
- [Spiking PointNet: Spiking Neural Networks for Point Clouds](https://mlsanity.com/paper/2310.06232/) (2023)
- [EventDrop: data augmentation for event-based learning](https://mlsanity.com/paper/2106.05836/) (2021)
- [RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks](https://mlsanity.com/paper/2308.06787/) (2023)
- [Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning](https://mlsanity.com/paper/2109.01905/) (2021)
- [Gated Attention Coding for Training High-performance and Efficient Spiking Neural Networks](https://mlsanity.com/paper/2308.06582/) (2023)
- [Fully Spiking Variational Autoencoder](https://mlsanity.com/paper/2110.00375/) (2021)
- [Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Network](https://mlsanity.com/paper/2401.01912/) (2024)
- [Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks](https://mlsanity.com/paper/2401.04486/) (2024)
- [Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN](https://mlsanity.com/paper/2403.03409/) (2024)

## Built on by (most cited first, 2 of 2)

- [Rethinking Spiking Neural Networks from an Ensemble Learning Perspective](https://mlsanity.com/paper/2502.14218/) (2025)
- [Temporal Regularization Training: Unleashing the Potential of Spiking Neural Networks](https://mlsanity.com/paper/2506.19256/) (2025)

## Abstract (arXiv)

Spiking neural networks (SNNs) have received widespread attention as an ultra-low power computing paradigm. Recent studies have shown that SNNs suffer from severe overfitting, which limits their generalization performance. In this paper, we propose a simple yet effective Temporal Reversal Regularization (TRR) to mitigate overfitting during training and facilitate generalization of SNNs. We exploit the inherent temporal properties of SNNs to perform input/feature temporal reversal perturbations, prompting the SNN to produce original-reversed consistent outputs and learn perturbation-invariant representations. To further enhance generalization, we utilize the lightweight ``star operation" (Hadamard product) for feature hybridization of original and temporally reversed spike firing rates, which expands the implicit dimensionality and acts as a spatio-temporal regularizer. We show theoretically that our method is able to tighten the upper bound of the generalization error, and extensive experiments on static/neuromorphic recognition as well as 3D point cloud classification tasks demonstrate its effectiveness, versatility, and adversarial robustness. In particular, our regularization significantly improves the recognition accuracy of low-latency SNN for neuromorphic objects, contributing to the real-world deployment of neuromorphic computational software-hardware integration.