# Masked Event Modeling: Self-Supervised Pretraining for Event Cameras

arXiv: https://arxiv.org/abs/2212.10368 · HTML: https://mlsanity.com/paper/2212.10368/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 79 references that are in the map.
- First posted on arXiv 20 Dec 2022.

## Builds on (most cited first, 30 of 57)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers](https://mlsanity.com/paper/2106.10270/) (2021)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)

## Built on by (most cited first, 30 of 41)

- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers](https://mlsanity.com/paper/2307.04129/) (2023)
- [EventCLIP: Adapting CLIP for Event-based Object Recognition](https://mlsanity.com/paper/2306.06354/) (2023)
- [EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding](https://mlsanity.com/paper/2308.03135/) (2023)
- [Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events](https://mlsanity.com/paper/2308.09383/) (2023)
- [LEOD: Label-Efficient Object Detection for Event Cameras](https://mlsanity.com/paper/2311.17286/) (2023)
- [EventVL: Understand Event Streams via Multimodal Large Language Model](https://mlsanity.com/paper/2501.13707/) (2025)
- [FlexEvent: Towards Flexible Event-Frame Object Detection at Varying Operational Frequencies](https://mlsanity.com/paper/2412.06708/) (2024)
- [EventFly: Event Camera Perception from Ground to the Sky](https://mlsanity.com/paper/2503.19916/) (2025)
- [Data-efficient Event Camera Pre-training via Disentangled Masked Modeling](https://mlsanity.com/paper/2403.00416/) (2024)
- [MambaPupil: Bidirectional Selective Recurrent model for Event-based Eye tracking](https://mlsanity.com/paper/2404.12083/) (2024)
- [EZSR: Event-based Zero-Shot Recognition](https://mlsanity.com/paper/2407.21616/) (2024)
- [CEIA: CLIP-Based Event-Image Alignment for Open-World Event-Based Understanding](https://mlsanity.com/paper/2407.06611/) (2024)
- [LLaFEA: Frame-Event Complementary Fusion for Fine-Grained Spatiotemporal Understanding in LMMs](https://mlsanity.com/paper/2503.06934/) (2025)
- [CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework](https://mlsanity.com/paper/2504.12576/) (2025)
- [TESPEC: Temporally-Enhanced Self-Supervised Pretraining for Event Cameras](https://mlsanity.com/paper/2508.00913/) (2025)
- [Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systems](https://mlsanity.com/paper/2512.24385/) (2025)
- [Generative Event Pretraining with Foundation Model Alignment](https://mlsanity.com/paper/2603.23032/) (2026)
- [Event Camera Guided Visual Media Restoration & 3D Reconstruction: A Survey](https://mlsanity.com/paper/2509.09971/) (2025)
- [EventFlash: Towards Efficient MLLMs for Event-Based Vision](https://mlsanity.com/paper/2602.03230/) (2026)
- [A Double Deep Learning-based Solution for Efficient Event Data Coding and Classification](https://mlsanity.com/paper/2407.15531/) (2024)
- [Pushing the boundaries of event subsampling in event-based video classification using CNNs](https://mlsanity.com/paper/2409.08953/) (2024)
- [Event USKT : U-State Space Model in Knowledge Transfer for Event Cameras](https://mlsanity.com/paper/2411.15276/) (2024)
- [Deep Learning-based Event Data Coding: A Joint Spatiotemporal and Polarity Solution](https://mlsanity.com/paper/2502.03285/) (2025)
- [Maximizing Asynchronicity in Event-based Neural Networks](https://mlsanity.com/paper/2505.11165/) (2025)
- [Making Every Event Count: Balancing Data Efficiency and Accuracy in Event Camera Subsampling](https://mlsanity.com/paper/2505.21187/) (2025)
- [Revealing Latent Information: A Physics-inspired Self-supervised Pre-training Framework for Noisy and Sparse Events](https://mlsanity.com/paper/2508.05507/) (2025)
- [Scaling Dense Event-Stream Pretraining from Visual Foundation Models](https://mlsanity.com/paper/2603.03969/) (2026)
- [PASS: Path-selective State Space Model for Event-based Recognition](https://mlsanity.com/paper/2409.16953/) (2024)

## Abstract (arXiv)

Event cameras asynchronously capture brightness changes with low latency, high temporal resolution, and high dynamic range. However, annotation of event data is a costly and laborious process, which limits the use of deep learning methods for classification and other semantic tasks with the event modality. To reduce the dependency on labeled event data, we introduce Masked Event Modeling (MEM), a self-supervised framework for events. Our method pretrains a neural network on unlabeled events, which can originate from any event camera recording. Subsequently, the pretrained model is finetuned on a downstream task, leading to a consistent improvement of the task accuracy. For example, our method reaches state-of-the-art classification accuracy across three datasets, N-ImageNet, N-Cars, and N-Caltech101, increasing the top-1 accuracy of previous work by significant margins. When tested on real-world event data, MEM is even superior to supervised RGB-based pretraining. The models pretrained with MEM are also label-efficient and generalize well to the dense task of semantic image segmentation.