mlsanityData to 14 Oct 2026

Decision SpikeFormer: Spike-Driven Transformer for Decision Making

arXiv 2504.03800 ↗First posted 4 Apr 2025cs.LG cs.AI cs.NEcited by 1

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

also in 1 more
Abstract (arXiv)

Offline reinforcement learning (RL) enables policy training solely on pre-collected data, avoiding direct environment interaction - a crucial benefit for energy-constrained embodied AI applications. Although Artificial Neural Networks (ANN)-based methods perform well in offline RL, their high computational and energy demands motivate exploration of more efficient alternatives. Spiking Neural Networks (SNNs) show promise for such tasks, given their low power consumption. In this work, we introduce DSFormer, the first spike-driven transformer model designed to tackle offline RL via sequence modeling. Unlike existing SNN transformers focused on spatial dimensions for vision tasks, we develop Temporal Spiking Self-Attention (TSSA) and Positional Spiking Self-Attention (PSSA) in DSFormer to capture the temporal and positional dependencies essential for sequence modeling in RL. Additionally, we propose Progressive Threshold-dependent Batch Normalization (PTBN), which combines the benefits of LayerNorm and BatchNorm to preserve temporal dependencies while maintaining the spiking nature of SNNs. Comprehensive results in the D4RL benchmark show DSFormer's superiority over both SNN and ANN counterparts, achieving 78.4% energy savings, highlighting DSFormer's advantages not only in energy efficiency but also in competitive performance. Code and models are public at https://wei-nijuan.github.io/DecisionSpikeFormer.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 56 of its 74 references that are in the map. First posted on arXiv 4 Apr 2025.

Builds on

56 of 74 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Q-value Regularized Transformer for Offline Reinforcement Learning27 May
  2. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learning21 Dec
  3. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making4 Oct
  4. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targets24 Jun
  5. 2022How Crucial is Transformer in Decision Transformer?26 Nov
  6. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  8. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  9. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  10. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  11. 2018Behavioral Cloning from Observationcited by 492
  12. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 406
37 more
  1. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  2. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  3. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 271
  4. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 238
  5. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 215
  6. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 198
  7. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 185
  8. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 182
  9. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 177
  10. 2023Spike-driven Transformercited by 146
  11. 2022MetaFormer Baselines for Visioncited by 132
  12. 2021An Attention Free Transformercited by 112
  13. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learningcited by 112
  14. 2024Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chipscited by 103
  15. 2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learningcited by 91
  16. 2023SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networkscited by 88
  17. 2021Advancing Spiking Neural Networks towards Deep Residual Learningcited by 79
  18. 2023Deep Directly-Trained Spiking Neural Networks for Object Detectioncited by 77
  19. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 76
  20. 2023Elastic Decision Transformercited by 67
  21. 2024Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detectioncited by 67
  22. 2024QKFormer: Hierarchical Spiking Transformer using Q-K Attentioncited by 49
  23. 2024SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanismscited by 31
  24. 2023RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networkscited by 28
  25. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  26. 2020Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networkscited by 23
  27. 2023Prompt-Tuning Decision Transformer with Preference Rankingcited by 19
  28. 2023Large Sequence Models for Sequential Decision-Making: A Surveycited by 19
  29. 2023SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillationcited by 17
  30. 2024SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalizationcited by 17
  31. 2022Spike Calibration: Fast and Accurate Conversion of Spiking Neural Network for Object Detection and Segmentationcited by 15
  32. 2023Enhancing the Performance of Transformer-based Spiking Neural Networks by SNN-optimized Downsampling with Precise Gradient Backpropagationcited by 10
  33. 2023Iteratively Refined Behavior Regularization for Offline Reinforcement Learningcited by 10
  34. 2024Spiking Convolutional Neural Networks for Text Classificationcited by 10
  35. 2021Quantile Filtered Imitation Learningcited by 6
  36. 2024Directly Training Temporal Spiking Neural Network with Sparse Surrogate Gradientcited by 6
  37. 2024Fourier Controller Networks for Real-Time Decision-Making in Embodied Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Edge Intelligence with Spiking Neural Networkscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Spiking Decision Transformers: Local Plasticity, Phase-Coding, and Dendritic Routing for Low-Power Sequence Control
  2. 2024Pre-trained Language Models Improve the Few-shot Prompt Ability of Decision Transformer
  3. 2025Energy-Efficient Deep Reinforcement Learning with Spiking Transformers
  4. 2023Recurrent Action Transformer with Memory
  5. 2025Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks
  6. 2026VL2Spike: Spike-driven Distillation from VLMs for Low-Power Visual Perception in Embodied AI
  7. 2024GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL
  8. 2023Multi-Objective Decision Transformers for Offline Reinforcement Learning