mlsanityData to 7 Oct 2026

Spiking Convolutional Neural Networks for Text Classification

arXiv 2406.19230 ↗First posted 27 Jun 2024cs.NE cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Spiking neural networks (SNNs) offer a promising pathway to implement deep neural networks (DNNs) in a more energy-efficient manner since their neurons are sparsely activated and inferences are event-driven. However, there have been very few works that have demonstrated the efficacy of SNNs in language tasks partially because it is non-trivial to represent words in the forms of spikes and to deal with variable-length texts by SNNs. This work presents a "conversion + fine-tuning" two-step method for training SNNs for text classification and proposes a simple but effective way to encode pre-trained word embeddings as spike trains. We show empirically that after fine-tuning with surrogate gradients, the converted SNNs achieve comparable results to their DNN counterparts with much less energy consumption across multiple datasets for both English and Chinese. We also show that such SNNs are more robust to adversarial attacks than DNNs.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 34 of its 63 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

34 of 63 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  7. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  8. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  9. 2018Generating Natural Language Adversarial Examplescited by 422
  10. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  11. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  12. 2018Deep Learning in Spiking Neural Networkscited by 258
22 more
  1. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 257
  2. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  3. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  4. 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 179
  5. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 168
  6. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 142
  7. 2017Deep Text Classification Can be Fooledcited by 105
  8. 2015Spiking Deep Networks with LIF Neuronscited by 63
  9. 2017Towards Crafting Text Adversarial Samplescited by 55
  10. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 55
  11. 2018Analogical Reasoning on Chinese Morphological and Semantic Relationscited by 50
  12. 2017Gradient Descent for Spiking Neural Networkscited by 47
  13. 2016Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardwarecited by 39
  14. 2017DART: Distribution Aware Retinal Transform for Event-based Camerascited by 39
  15. 2019Natural Language Adversarial Defense through Synonym Encodingcited by 39
  16. 2020Surrogate gradients for analog neuromorphic computingcited by 38
  17. 2020Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applicationscited by 34
  18. 2021A Long Short-Term Memory for AI Applications in Spike-based Neuromorphic Hardwarecited by 32
  19. 2017Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networkscited by 31
  20. 2021Certified Robustness to Programmable Transformations in LSTMscited by 18
  21. 2020On the Transferability of Adversarial Attacksagainst Neural Text Classifiercited by 16
  22. 2017DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 3.

10 papers.

  1. 2024SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Networkcited by 3
  2. 2025Improving Performance of Spike-based Deep Q-Learning using Ternary Neuronscited by 3
  3. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  4. 2025New Spiking Architecture for Multi-Modal Decision-Making in Autonomous Vehiclescited by 2
  5. 2025FAS: Fast ANN-SNN Conversion for Spiking Large Language Modelscited by 1
  6. 2025Decision SpikeFormer: Spike-Driven Transformer for Decision Makingcited by 1
  7. 2026Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrievalcited by 1
  8. 2026Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradientscited by 1
  9. 2025TDFormer: A Top-Down Attention-Controlled Spiking Transformercited by 0
  10. 2026Efficient Aspect Term Extraction using Spiking Neural Networkcited by 0