# Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation

arXiv: https://arxiv.org/abs/2609.16811 · HTML: https://mlsanity.com/paper/2609.16811/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception](https://mlsanity.com/paper/1801.10202/) (2018)
- [UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks](https://mlsanity.com/paper/2105.00956/) (2021)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [Your ViT is Secretly an Image Segmentation Model](https://mlsanity.com/paper/2503.19108/) (2025)
- [Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric](https://mlsanity.com/paper/2211.11010/) (2022)
- [OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2405.05259/) (2024)
- [Event Camera Data Dense Pre-training](https://mlsanity.com/paper/2311.11533/) (2023)
- [Event-based Tiny Object Detection: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2506.23575/) (2025)
- [EZSR: Event-based Zero-Shot Recognition](https://mlsanity.com/paper/2407.21616/) (2024)
- [SEE: See Everything Every Time -- Adaptive Brightness Adjustment for Broad Light Range Images via Events](https://mlsanity.com/paper/2502.21120/) (2025)
- [CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework](https://mlsanity.com/paper/2504.12576/) (2025)
- [Generative Event Pretraining with Foundation Model Alignment](https://mlsanity.com/paper/2603.23032/) (2026)
- [Scaling Dense Event-Stream Pretraining from Visual Foundation Models](https://mlsanity.com/paper/2603.03969/) (2026)
- [Rethinking Event-Based Object Detection through Representation-Level Temporal Aggregation and Model-Level Hypergraph Reasoning](https://mlsanity.com/paper/2605.08825/) (2026)

## Abstract (arXiv)

Event cameras have shown great potential for robust visual perception, yet scaling event representation learning remains challenging due to the scarcity of large-scale annotated event data. Pretrained image models provide scalable semantic supervision, but existing image-to-event methods rely on rigid pixel-wise or token-wise alignment that overlooks modality discrepancies in texture, density, and appearance, potentially causing semantic collapse and limiting transferability. To address this issue, we propose Hyper-RED, a simple, painless, and scalable image-to-event pretraining framework that transfers high-order semantic structures from images to events. Hyper-RED uses hypergraphs to model and align high-order semantic associations among multiple image and event tokens, enabling cross-modal knowledge transfer while accommodating modality-specific differences rather than enforcing rigid one-to-one correspondence. Specifically, given a paired event--image sample, Hyper-RED leverages DINOv3 to extract spatial token representations and constructs image, event, and cross-modal semantic hypergraphs, where each hyperedge connects multiple semantically correlated tokens. We further introduce a hypergraph relational distillation loss that imposes complementary intra- and cross-modal constraints, enabling the event encoder to inherit image-derived semantic organization while preserving local relational consistency and event-specific characteristics. Experiments on three tasks across five event datasets demonstrate consistent scaling from ViT-S to ViT-L and state-of-the-art performance (Fig.1). The code is available at: https://github.com/meisenwang/Hyper--RED.