# Complex Dynamic Neurons Improved Spiking Transformer Network for Efficient Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2302.01194 · HTML: https://mlsanity.com/paper/2302.01194/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2023.

## Built on by (most cited first, 20 of 20)

- [Spike-driven Transformer](https://mlsanity.com/paper/2307.01694/) (2023)
- [Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips](https://mlsanity.com/paper/2404.03663/) (2024)
- [Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training](https://mlsanity.com/paper/2411.16061/) (2024)
- [Spiking Vision Transformer with Saccadic Attention](https://mlsanity.com/paper/2502.12677/) (2025)
- [Towards High-performance Spiking Transformers from ANN to SNN Conversion](https://mlsanity.com/paper/2502.21193/) (2025)
- [Spiking Tucker Fusion Transformer for Audio-Visual Zero-Shot Learning](https://mlsanity.com/paper/2407.08130/) (2024)
- [Ternary Spike-based Neuromorphic Signal Processing System](https://mlsanity.com/paper/2407.05310/) (2024)
- [ODE-based Recurrent Model-free Reinforcement Learning for POMDPs](https://mlsanity.com/paper/2309.14078/) (2023)
- [AutoST: Training-free Neural Architecture Search for Spiking Transformers](https://mlsanity.com/paper/2307.00293/) (2023)
- [TS-LIF: A Temporal Segment Spiking Neuron Network for Time Series Forecasting](https://mlsanity.com/paper/2503.05108/) (2025)
- [Spiking Variational Graph Representation Inference for Video Summarization](https://mlsanity.com/paper/2508.15389/) (2025)
- [Attention-free Spikformer: Mixing Spike Sequences with Simple Linear Transforms](https://mlsanity.com/paper/2308.02557/) (2023)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [IML-Spikeformer: Input-aware Multi-Level Spiking Transformer for Speech Processing](https://mlsanity.com/paper/2507.07396/) (2025)
- [Multi-Modal Machine Learning for Assessing Gaming Skills in Online Streaming: A Case Study with CS:GO](https://mlsanity.com/paper/2307.12236/) (2023)
- [Learning Spiking Neural Network from Easy to Hard task](https://mlsanity.com/paper/2309.04737/) (2023)
- [Fourier or Wavelet bases as counterpart self-attention in spikformer for efficient visual classification](https://mlsanity.com/paper/2403.18228/) (2024)
- [Temporal Misalignment in ANN-SNN Conversion and Its Mitigation via Probabilistic Spiking Neurons](https://mlsanity.com/paper/2502.14487/) (2025)
- [Toward Efficient Spiking Transformers: Synapse Pruning Meets Synergistic Learning-Based Compensation](https://mlsanity.com/paper/2508.01992/) (2025)
- [DVA-Neurons: Design and Verification of Adaptive LIF Neurons: From Single-Neuron Dynamics to Multi-Neuron Spiking Networks](https://mlsanity.com/paper/2609.22775/) (2026)

## Abstract (arXiv)

The spiking neural network (SNN) using leaky-integrated-and-fire (LIF) neurons has been commonly used in automatic speech recognition (ASR) tasks. However, the LIF neuron is still relatively simple compared to that in the biological brain. Further research on more types of neurons with different scales of neuronal dynamics is necessary. Here we introduce four types of neuronal dynamics to post-process the sequential patterns generated from the spiking transformer to get the complex dynamic neuron improved spiking transformer neural network (DyTr-SNN). We found that the DyTr-SNN could handle the non-toy automatic speech recognition task well, representing a lower phoneme error rate, lower computational cost, and higher robustness. These results indicate that the further cooperation of SNNs and neural dynamics at the neuron and network scales might have much in store for the future, especially on the ASR tasks.