mlsanityData to 14 Oct 2026

Event-Frame Fusion for Inter-Frame Segmentation via Event-Guided Motion

arXiv 2609.22500 ↗First posted 18 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Autonomous navigation requires precise and efficient semantic segmentation, yet existing frame-based approaches remain limited by motion blur, glare, latency, and the low temporal resolution (20-30 FPS) of conventional cameras, which leads to information loss between frames. Event cameras have emerged as an alternative sensing modality, capturing intensity changes asynchronously with high temporal resolution, high dynamic range, and sparse outputs. However, event-based algorithms still fall short of frame-based ones in accuracy, as most segmentation methods are designed for dense frame data. To overcome these limitations, we propose a hybrid vision architecture that combines conventional frame-based and event-based cameras. The system integrates two complementary components: (1) a compact Spiking Neural Network (SNN) with 42k parameters for motion estimation, and (2) a lightweight event-driven SNN with 0.84M parameters for frame-based semantic segmentation, which interpolates motion between frames to refine segmentation results. By predicting inter-frame segmentations, the framework achieves segmentation rates of up to 500 Hz with an energy consumption below 1.87 mJ per inference, while maintaining real-time GPU execution at frequencies up to 200 Hz. Additionally, our approach compensates for information loss in frames affected by blur or overexposure, enabling more robust perception in challenging conditions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

33 of 43 references.

References · 33

  1. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  2. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  3. 2019Event-based Vision: A Surveycited by 1,016
  4. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  5. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  6. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 233
  7. 2018EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Camerascited by 224
  8. 2018Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Carscited by 200
  9. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
  10. 2019Video to Events: Recycling Video Datasets for Event Camerascited by 165
  11. 2021E-RAFT: Dense Optical Flow from Event Camerascited by 152
  12. 2020Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancycited by 125
21 more
  1. 2021Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networkscited by 99
  2. 2018Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networkscited by 97
  3. 2018EV-SegNet: Semantic Segmentation for Event-based Camerascited by 93
  4. 2022ESS: Learning Event-based Semantic Segmentation from Still Imagescited by 82
  5. 2018Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Videocited by 78
  6. 2018Low-Latency Video Semantic Segmentationcited by 61
  7. 2021Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentationcited by 55
  8. 2018Dynamic Video Segmentation Networkcited by 51
  9. 2021EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillationcited by 47
  10. 2022Dense Continuous-Time Optical Flow from Events and Framescited by 46
  11. 2023Taming Contrast Maximization for Learning Sequential, Low-latency, Event-based Optical Flowcited by 39
  12. 2022An Analytical Estimation of Spiking Neural Networks Energy Efficiencycited by 38
  13. 2023Optical flow estimation from event-based cameras and spiking neural networkscited by 33
  14. 2021Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translationcited by 32
  15. 2023TMA: Temporal Motion Aggregation for Event-based Optical Flowcited by 32
  16. 2022Lightweight Event-based Optical Flow Estimation via Iterative Deblurringcited by 31
  17. 2021A Spiking Neural Network for Image Segmentationcited by 27
  18. 2022HALSIE: Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalitiescited by 25
  19. 2023Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Networkcited by 9
  20. 2024EvSegSNN: Neuromorphic Semantic Segmentation for Event Datacited by 7
  21. 2025Enhanced Neuromorphic Semantic Segmentation Latency through Stream Eventcited by 1

Built on by

Not cited yet.