mlsanityData to 14 Oct 2026

Event-based Action Recognition by Pre-Training on Multi-Frame Representations

arXiv 2610.13856 ↗First posted 9 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Action recognition is a crucial task in domains like robotics, patient monitoring, or surveillance. Recently, event-based action recognition has attracted much attention due to the event camera's ability to capture fast motions and its improved privacy over conventional vision cameras. While frame-based event representations work well for short event sequences, their discriminative ability declines sharply with longer and more complex event sequences. In this work, we introduce a multi-frame representation that greatly improves recognition performance over its single-frame counterpart. We adopt a masked auto-encoding framework to pre-train a Vision Transformer using a sparsity-aware masking algorithm and token selection mechanism to improve downstream performance and control computational complexity. We also introduce multiple mask tokens for the decoder with a position-aware attention mechanism, leading to further improved performance. Our proposed method shows impressive performance for challenging long-duration and complex event sequences, yielding state-of-the-art performance on THU-EACT-50 CHL and DailyDVS-200.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

31 of 42 references.

References · 31

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  4. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  5. 2019Event-based Vision: A Surveycited by 1,016
  6. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  7. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  8. 2021Video Transformer Networkcited by 272
  9. 2018Human Action Recognition and Prediction: A Surveycited by 227
  10. 2018Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Carscited by 200
  11. 2018Event-based Moving Object Detection and Trackingcited by 127
  12. 2020Temporal Binary Representation for Event-Based Action Recognitioncited by 51
19 more
  1. 2022Event Transformer. A sparse-aware solution for efficient event data processingcited by 50
  2. 2023Event Camera Data Pre-trainingcited by 50
  3. 2022HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensorscited by 47
  4. 2022AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoderscited by 43
  5. 2022Masked Event Modeling: Self-Supervised Pretraining for Event Camerascited by 41
  6. 2023EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understandingcited by 34
  7. 2024Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMambacited by 19
  8. 2024DailyDVS-200: A Comprehensive Benchmark Dataset for Event-Based Action Recognitioncited by 17
  9. 2022SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-trainingcited by 15
  10. 2024Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithmscited by 12
  11. 2023EventTransAct: A video transformer-based framework for Event-camera based action recognitioncited by 11
  12. 2024Data-efficient Event Camera Pre-training via Disentangled Masked Modelingcited by 9
  13. 2025Event Masked Autoencoder: Point-wise Action Recognition with Event-Based Camerascited by 4
  14. 2026Generative Event Pretraining with Foundation Model Alignmentcited by 4
  15. 2025Revealing Latent Information: A Physics-inspired Self-supervised Pre-training Framework for Noisy and Sparse Eventscited by 2
  16. 2026DarkShake-DVS: Event-based Human Action Recognition under Low-light andShaking Camera Conditionscited by 2
  17. 2024PASS: Path-selective State Space Model for Event-based Recognitioncited by 1
  18. 2026SMV-EAR: Bring Spatiotemporal Multi-View Representation Learning into Efficient Event-Based Action Recognitioncited by 1
  19. 2026HoloEv-Net: Efficient Event-based Action Recognition via Holographic Spatial Embedding and Global Spectral Gatingcited by 1

Built on by

Not cited yet.