Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing
Not on the map. Why?
Abstract (arXiv)
Fusing standard RGB frames with asynchronous event streams has emerged as a definitive paradigm for robust perception in degraded environments. Although unified backbones have recently gained traction in multi-modal vision, adapting them to the RGB-Event domain remains fundamentally challenging. Existing architectures either resort to decoupled dual encoders that double computational overhead, or adopt generic unified designs that fail to resolve implicit geometric parallax and cross-spectral aliasing under the extreme representational divide between dense intensity grids and sparse kinematic spikes. To transcend these bottlenecks, we present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing. To achieve profound modal synergy, Evita explicitly embeds a suite of intrinsic co-learning modules directly into every encoder layer. Specifically, it features Geometric Parallax Rectification for adaptive spatial alignment, Harmonic Spectral Resonance for texture transfer exclusively in the complex frequency domain, and Transient Global Routing for event-driven asymmetric attention. To guarantee robust feature extraction against spatial misalignments and decouple representations from specific event encodings, we construct N-ImageNetV2 alongside a stochastic event representation mixing pretraining protocol, empowering the network to seamlessly accommodate arbitrary event formats in downstream tasks. Extensive evaluations across the DELIVER, DDD17, and DSEC benchmarks confirm that Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.The code are publicly available at: https://github.com/chaineypung/Evita.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 65 references that are in the map. First posted on arXiv 10 Jul 2026.
Builds on
49 of 65 references.
References · 49
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
- 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
- 2019Event-based Vision: A Surveycited by 1,016
- 2021Multi-Stage Progressive Image Restorationcited by 816
- 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 670
- 2023LightGlue: Local Feature Matching at Light Speedcited by 590
- 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
- 2020Zero-Reference Deep Curve Estimation for Low-Light Image Enhancementcited by 542
37 more
- 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 541
- 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
- 2021Rethinking Coarse-to-Fine Approach in Single Image Deblurringcited by 326
- 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 303
- 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
- 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
- 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 231
- 2022Omnivore: A Single Model for Many Visual Modalitiescited by 215
- 2021TimeLens: Event-based Video Frame Interpolationcited by 151
- 2022Multimodal Token Fusion for Vision Transformerscited by 136
- 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 117
- 2020Event-based Stereo Visual Odometrycited by 105
- 2018Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networkscited by 97
- 2018EV-SegNet: Semantic Segmentation for Event-based Camerascited by 93
- 2021N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Camerascited by 92
- 2023Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarkscited by 90
- 2020Efficient RGB-D Semantic Segmentation for Indoor Scene Analysiscited by 84
- 2022ESS: Learning Event-based Semantic Segmentation from Still Imagescited by 82
- 2023DFormer: Rethinking RGBD Representation Learning for Semantic Segmentationcited by 60
- 2022Unifying Motion Deblurring and Frame Interpolation with Eventscited by 58
- 2024Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approachcited by 48
- 2024GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformercited by 44
- 2024StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentationcited by 39
- 2022Pushing the Limits of Asynchronous Graph-based Object Detection with Event Camerascited by 32
- 2022HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detectioncited by 31
- 2025DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentationcited by 31
- 2022Generalized Differentiable RANSACcited by 30
- 2022HALSIE: Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalitiescited by 25
- 2020Learning to Deblur and Generate High Frame Rate Video with an Event Cameracited by 22
- 2023Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalizationcited by 14
- 2023Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Networkcited by 9
- 2025Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approachcited by 6
- 2025OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentationcited by 5
- 2025MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentationcited by 4
- 2025EvRWKV: A Continuous Interactive RWKV Framework for Effective Event-Guided Low-Light Image Enhancementcited by 2
- 2025TUNI: Unifying Pre-training and Fine-tuning with Modality-Aware Mutual Learning and Rectification for RGB-T Semantic Segmentationcited by 2
- 2025Re-coding for Uncertainties: Edge-awareness Semantic Concordance for Resilient Event-RGB Segmentationcited by 1
Built on by
Not cited yet.