mlsanityData to 14 Oct 2026

Event USKT : U-State Space Model in Knowledge Transfer for Event Cameras

arXiv 2411.15276 ↗First posted 22 Nov 2024cs.CV cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Event cameras, as an emerging imaging technology, offer distinct advantages over traditional RGB cameras, including reduced energy consumption and higher frame rates. However, the limited quantity of available event data presents a significant challenge, hindering their broader development. To alleviate this issue, we introduce a tailored U-shaped State Space Model Knowledge Transfer (USKT) framework for Event-to-RGB knowledge transfer. This framework generates inputs compatible with RGB frames, enabling event data to effectively reuse pre-trained RGB models and achieve competitive performance with minimal parameter tuning. Within the USKT architecture, we also propose a bidirectional reverse state space model. Unlike conventional bidirectional scanning mechanisms, the proposed Bidirectional Reverse State Space Model (BiR-SSM) leverages a shared weight strategy, which facilitates efficient modeling while conserving computational resources. In terms of effectiveness, integrating USKT with ResNet50 as the backbone improves model performance by 0.95%, 3.57%, and 2.9% on DVS128 Gesture, N-Caltech101, and CIFAR-10-DVS datasets, respectively, underscoring USKT's adaptability and effectiveness. The code will be made available upon acceptance.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 40 of its 79 references that are in the map. First posted on arXiv 22 Nov 2024.

Builds on

40 of 79 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  5. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  6. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  7. 2019Event-based Vision: A Surveycited by 1,016
  8. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  9. 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 314
  10. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  11. 2018HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classificationcited by 218
  12. 2018A Variational U-Net for Conditional Appearance and Shape Generationcited by 124
28 more
  1. 2019MineGAN: effective knowledge transfer from GANs to target domains with few imagescited by 106
  2. 2021Temporal-wise Attention Spiking Neural Networks for Event Streams Classificationcited by 106
  3. 2019Knowledge Adaptation for Efficient Semantic Segmentationcited by 105
  4. 2021N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Camerascited by 92
  5. 2018CoNet: Collaborative Cross Networks for Cross-Domain Recommendationcited by 81
  6. 2022Visual Prompt Tuning for Generative Transfer Learningcited by 75
  7. 2023Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baselinecited by 73
  8. 2017Continuous-Time Visual-Inertial Odometry for Event Camerascited by 57
  9. 2021DINE: Domain Adaptation from Single and Multiple Black-box Predictorscited by 51
  10. 2023Graph-based Asynchronous Event Processing for Rapid Object Recognitioncited by 51
  11. 2023Event Camera Data Pre-trainingcited by 50
  12. 2022Masked Event Modeling: Self-Supervised Pretraining for Event Camerascited by 41
  13. 2021A Voxel Graph CNN for Object Classification with Event Camerascited by 38
  14. 2020Learning Affinity-Aware Upsampling for Deep Image Mattingcited by 37
  15. 2023Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolutioncited by 33
  16. 2021Real-Time Optical Flow for Vehicular Perception with Low- and High-Resolution Event Camerascited by 27
  17. 2022Exploiting Spatial Sparsity for Event Cameras with Visual Transformerscited by 23
  18. 2021BackEISNN: A Deep Spiking Neural Network with Adaptive Self-Feedback and Balanced Excitatory-Inhibitory Neuronscited by 22
  19. 2022Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networkscited by 21
  20. 2023Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operatorcited by 20
  21. 2022Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Networkcited by 19
  22. 2024Hypergraph-based Multi-View Action Recognition using Event Camerascited by 19
  23. 2020Structure-Aware Network for Lane Marker Extraction with Dynamic Vision Sensorcited by 9
  24. 2024PathoTune: Adapting Visual Foundation Model to Pathological Specialistscited by 8
  25. 2020TriGAN: Image-to-Image Translation for Multi-Source Domain Adaptationcited by 7
  26. 2023Learning Bottleneck Transformer for Event Image-Voxel Feature Fusion based Classificationcited by 6
  27. 2023Object-centric Cross-modal Feature Distillation for Event-based Object Detectioncited by 6
  28. 2022Transfer Learning with Pre-trained Conditional Generative Modelscited by 4

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025MemMamba: Rethinking Memory Patterns in State Space Modelcited by 3
  2. 2025Hypergraph Mamba for Efficient Whole Slide Image Understandingcited by 1