mlsanityData to 14 Oct 2026

Embracing Events and Frames with Hierarchical Feature Refinement Network for Object Detection

arXiv 2407.12582 ↗First posted 17 Jul 2024cs.CV cs.AI cs.ROcited by 28

Not on the map. Why?

Abstract (arXiv)

In frame-based vision, object detection faces substantial performance degradation under challenging conditions due to the limited sensing capability of conventional cameras. Event cameras output sparse and asynchronous events, providing a potential solution to solve these problems. However, effectively fusing two heterogeneous modalities remains an open issue. In this work, we propose a novel hierarchical feature refinement network for event-frame fusion. The core concept is the design of the coarse-to-fine fusion module, denoted as the cross-modality adaptive feature refinement (CAFR) module. In the initial phase, the bidirectional cross-modality interaction (BCI) part facilitates information bridging from two distinct sources. Subsequently, the features are further refined by aligning the channel-level mean and variance in the two-fold adaptive feature refinement (TAFR) part. We conducted extensive experiments on two benchmarks: the low-resolution PKU-DDD17-Car dataset and the high-resolution DSEC dataset. Experimental results show that our method surpasses the state-of-the-art by an impressive margin of 8.0% on the DSEC dataset. Besides, our method exhibits significantly better robustness (69.5% versus 38.7%) when introducing 15 different corruption types to the frame images. The code can be found at the link (https://github.com/HuCaoFighting/FRN).

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

38 of 61 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  8. 2017Squeeze-and-Excitation Networkscited by 4,789
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  11. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  12. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
26 more
  1. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,630
  2. 2019Event-based Vision: A Surveycited by 1,016
  3. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 575
  4. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  5. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  6. 2018Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotioncited by 307
  7. 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 303
  8. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 231
  9. 2020Learning to Detect Objects with a 1 Megapixel Event Cameracited by 186
  10. 2022Recurrent Vision Transformers for Object Detection with Event Camerascited by 159
  11. 2020Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentationcited by 122
  12. 2016Understanding How Image Quality Affects Deep Neural Networkscited by 119
  13. 2019Imbalance Problems in Object Detection: A Reviewcited by 118
  14. 2021Event-Based Fusion for Motion Deblurring with Cross-modal Attentioncited by 113
  15. 2021Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Predictioncited by 90
  16. 2017DDD17: End-To-End DAVIS Driving Datasetcited by 84
  17. 2020Efficient RGB-D Semantic Segmentation for Indoor Scene Analysiscited by 84
  18. 2020JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detectioncited by 79
  19. 2024State Space Models for Event Camerascited by 75
  20. 2021Specificity-preserving RGB-D Saliency Detectioncited by 70
  21. 2020SOE-Net: A Self-Attention and Orientation Encoding Network for Point Cloud based Place Recognitioncited by 66
  22. 2021Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusioncited by 56
  23. 2022RGB-Event Fusion for Moving Object Detection in Autonomous Drivingcited by 40
  24. 2020Cascade Graph Neural Networks for RGB-D Salient Object Detectioncited by 36
  25. 2022CASSPR: Cross Attention Single Scan Place Recognitioncited by 22
  26. 2019FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videoscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 14; 2026: 10.

28 papers.

Sort:

By area:

  1. 2024FlexEvent: Towards Flexible Event-Frame Object Detection at Varying Operational Frequenciescited by 10
  2. 2025Adaptive Event Stream Slicing for Open-Vocabulary Event-Based Object Detection via Vision-Language Knowledge Distillationcited by 8
  3. 2024Frequency-Adaptive Low-Latency Object Detection Using Events and Framescited by 7
  4. 2025Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Applicationcited by 6
  5. 2025Talk2Event: Grounded Understanding of Dynamic Scenes from Event Camerascited by 6
  6. 2025Beyond conventional vision: RGB-event fusion for robust object detection in dynamic traffic scenarioscited by 6
  7. 2024BlinkTrack: Feature Tracking over 80 FPS via Events and Imagescited by 4
  8. 2025Unleashing the Temporal Potential of Stereo Event Cameras for Continuous-Time 3D Object Detectioncited by 4
  9. 2025URNet: Uncertainty-aware Refinement Network for Event-based Stereo Depth Estimationcited by 4
  10. 2025Event-aided Semantic Scene Completioncited by 3
  11. 2025Enhancing 3D Semantic Scene Completion with a Refinement Modulecited by 3
  12. 2024Steering Prediction via a Multi-Sensor System for Autonomous Racingcited by 2
16 more
  1. 2025EvAnimate: Event-conditioned Image-to-Video Generation for Human Animationcited by 2
  2. 2026EventDrive: Event Cameras for Vision-Language Driving Intelligencecited by 2
  3. 2025Dynamic Graph Induced Contour-aware Heat Conduction Network for Event-based Object Detectioncited by 1
  4. 2025WD-DETR: Wavelet Denoising-Enhanced Real-Time Object Detection Transformer for Robot Perception with Event Camerascited by 1
  5. 2025Focus Through Motion: RGB-Event Collaborative Token Sparsification for Efficient Object Detectioncited by 1
  6. 2025TUMTraf EMOT: Event-Based Multi-Object Tracking Dataset and Baseline for Traffic Scenarioscited by 1
  7. 2025BiXFormer: A Robust Framework for Maximizing Modality Effectiveness in Multi-Modal Semantic Segmentationcited by 0
  8. 2026RoEL: Robust Event-based 3D Line Reconstructioncited by 0
  9. 2026Energy-Aware Imitation Learning for Steering Prediction Using Events and Framescited by 0
  10. 2026SegRGB-X: General RGB-X Semantic Segmentation Modelcited by 0
  11. 2026CMTFormer: Marrying Transformer with Hierarchical Information Interaction for RGB-Event Object Detectioncited by 0
  12. 2026Neuromorphic Object Detection: An In-Depth Study and Future Directionscited by 0
  13. 2026FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learningcited by 0
  14. 2026Efficient and Robust Absolute Pose Estimation via Gravity-Prior-Driven Transformation Decoupling and Pose Refinementcited by 0
  15. 2026IT-TextFusion: Iterative Text-Image Interaction with Text-Guided Residual Refinement for Degradation-Aware Image Fusioncited by 0
  16. 2026Multimodal Fusion of RGB and Complementary Modalities for Semantic Segmentationcited by 0