mlsanityData to 14 Oct 2026

Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing

arXiv 2607.09143 ↗First posted 10 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Fusing standard RGB frames with asynchronous event streams has emerged as a definitive paradigm for robust perception in degraded environments. Although unified backbones have recently gained traction in multi-modal vision, adapting them to the RGB-Event domain remains fundamentally challenging. Existing architectures either resort to decoupled dual encoders that double computational overhead, or adopt generic unified designs that fail to resolve implicit geometric parallax and cross-spectral aliasing under the extreme representational divide between dense intensity grids and sparse kinematic spikes. To transcend these bottlenecks, we present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing. To achieve profound modal synergy, Evita explicitly embeds a suite of intrinsic co-learning modules directly into every encoder layer. Specifically, it features Geometric Parallax Rectification for adaptive spatial alignment, Harmonic Spectral Resonance for texture transfer exclusively in the complex frequency domain, and Transient Global Routing for event-driven asymmetric attention. To guarantee robust feature extraction against spatial misalignments and decouple representations from specific event encodings, we construct N-ImageNetV2 alongside a stochastic event representation mixing pretraining protocol, empowering the network to seamlessly accommodate arbitrary event formats in downstream tasks. Extensive evaluations across the DELIVER, DDD17, and DSEC benchmarks confirm that Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.The code are publicly available at: https://github.com/chaineypung/Evita.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 65 references that are in the map. First posted on arXiv 10 Jul 2026.

Builds on

49 of 65 references.

References · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  5. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  6. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  7. 2019Event-based Vision: A Surveycited by 1,016
  8. 2021Multi-Stage Progressive Image Restorationcited by 816
  9. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 670
  10. 2023LightGlue: Local Feature Matching at Light Speedcited by 590
  11. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  12. 2020Zero-Reference Deep Curve Estimation for Low-Light Image Enhancementcited by 542
37 more
  1. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 541
  2. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  3. 2021Rethinking Coarse-to-Fine Approach in Single Image Deblurringcited by 326
  4. 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 303
  5. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
  6. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  7. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 231
  8. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 215
  9. 2021TimeLens: Event-based Video Frame Interpolationcited by 151
  10. 2022Multimodal Token Fusion for Vision Transformerscited by 136
  11. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 117
  12. 2020Event-based Stereo Visual Odometrycited by 105
  13. 2018Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networkscited by 97
  14. 2018EV-SegNet: Semantic Segmentation for Event-based Camerascited by 93
  15. 2021N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Camerascited by 92
  16. 2023Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarkscited by 90
  17. 2020Efficient RGB-D Semantic Segmentation for Indoor Scene Analysiscited by 84
  18. 2022ESS: Learning Event-based Semantic Segmentation from Still Imagescited by 82
  19. 2023DFormer: Rethinking RGBD Representation Learning for Semantic Segmentationcited by 60
  20. 2022Unifying Motion Deblurring and Frame Interpolation with Eventscited by 58
  21. 2024Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approachcited by 48
  22. 2024GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformercited by 44
  23. 2024StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentationcited by 39
  24. 2022Pushing the Limits of Asynchronous Graph-based Object Detection with Event Camerascited by 32
  25. 2022HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detectioncited by 31
  26. 2025DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentationcited by 31
  27. 2022Generalized Differentiable RANSACcited by 30
  28. 2022HALSIE: Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalitiescited by 25
  29. 2020Learning to Deblur and Generate High Frame Rate Video with an Event Cameracited by 22
  30. 2023Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalizationcited by 14
  31. 2023Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Networkcited by 9
  32. 2025Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approachcited by 6
  33. 2025OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentationcited by 5
  34. 2025MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentationcited by 4
  35. 2025EvRWKV: A Continuous Interactive RWKV Framework for Effective Event-Guided Low-Light Image Enhancementcited by 2
  36. 2025TUNI: Unifying Pre-training and Fine-tuning with Modality-Aware Mutual Learning and Rectification for RGB-T Semantic Segmentationcited by 2
  37. 2025Re-coding for Uncertainties: Edge-awareness Semantic Concordance for Resilient Event-RGB Segmentationcited by 1

Built on by

Not cited yet.