# Revealing Latent Information: A Physics-inspired Self-supervised Pre-training Framework for Noisy and Sparse Events

arXiv: https://arxiv.org/abs/2508.05507 · HTML: https://mlsanity.com/paper/2508.05507/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 67 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Builds on (most cited first, 30 of 45)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Contrastive Masked Autoencoders are Stronger Vision Learners](https://mlsanity.com/paper/2207.13532/) (2022)
- [Discrete Variational Autoencoders](https://mlsanity.com/paper/1609.02200/) (2016)
- [Temporal-wise Attention Spiking Neural Networks for Event Streams Classification](https://mlsanity.com/paper/2107.11711/) (2021)
- [ConvMAE: Masked Convolution Meets Masked Autoencoders](https://mlsanity.com/paper/2205.03892/) (2022)
- [Event-based Asynchronous Sparse Convolutional Networks](https://mlsanity.com/paper/2003.09148/) (2020)
- [EV-SegNet: Semantic Segmentation for Event-based Cameras](https://mlsanity.com/paper/1811.12039/) (2018)
- [N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras](https://mlsanity.com/paper/2112.01041/) (2021)
- [MVP: Multimodality-guided Visual Pre-training](https://mlsanity.com/paper/2203.05175/) (2022)
- [DDD17: End-To-End DAVIS Driving Dataset](https://mlsanity.com/paper/1711.01458/) (2017)
- [ESS: Learning Event-based Semantic Segmentation from Still Images](https://mlsanity.com/paper/2203.10016/) (2022)
- [GET: Group Event Transformer for Event-Based Vision](https://mlsanity.com/paper/2310.02642/) (2023)
- [MILAN: Masked Image Pretraining on Language Assisted Representation](https://mlsanity.com/paper/2208.06049/) (2022)
- [Asynchronous Convolutional Networks for Object Detection in Neuromorphic Cameras](https://mlsanity.com/paper/1805.07931/) (2018)
- [LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing](https://mlsanity.com/paper/2011.06176/) (2020)

## Built on by (most cited first, 2 of 2)

- [Scaling Dense Event-Stream Pretraining from Visual Foundation Models](https://mlsanity.com/paper/2603.03969/) (2026)
- [Event-based Action Recognition by Pre-Training on Multi-Frame Representations](https://mlsanity.com/paper/2610.13856/) (2026)

## Abstract (arXiv)

Event camera, a novel neuromorphic vision sensor, records data with high temporal resolution and wide dynamic range, offering new possibilities for accurate visual representation in challenging scenarios. However, event data is inherently sparse and noisy, mainly reflecting brightness changes, which complicates effective feature extraction. To address this, we propose a self-supervised pre-training framework to fully reveal latent information in event data, including edge information and texture cues. Our framework consists of three stages: Difference-guided Masked Modeling, inspired by the event physical sampling process, reconstructs temporal intensity difference maps to extract enhanced information from raw event data. Backbone-fixed Feature Transition contrasts event and image features without updating the backbone to preserve representations learned from masked modeling and stabilizing their effect on contrastive learning. Focus-aimed Contrastive Learning updates the entire model to improve semantic discrimination by focusing on high-value regions. Extensive experiments show our framework is robust and consistently outperforms state-of-the-art methods on various downstream tasks, including object recognition, semantic segmentation, and optical flow estimation. The code and dataset are available at https://github.com/BIT-Vision/EventPretrain.