mlsanityData to 7 Oct 2026

Erasure-based Interaction Network for RGBT Video Object Detection and A Unified Benchmark

arXiv 2308.01630 ↗First posted 3 Aug 2023cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Recently, many breakthroughs are made in the field of Video Object Detection (VOD), but the performance is still limited due to the imaging limitations of RGB sensors in adverse illumination conditions. To alleviate this issue, this work introduces a new computer vision task called RGB-thermal (RGBT) VOD by introducing the thermal modality that is insensitive to adverse illumination conditions. To promote the research and development of RGBT VOD, we design a novel Erasure-based Interaction Network (EINet) and establish a comprehensive benchmark dataset (VT-VOD50) for this task. Traditional VOD methods often leverage temporal information by using many auxiliary frames, and thus have large computational burden. Considering that thermal images exhibit less noise than RGB ones, we develop a negative activation function that is used to erase the noise of RGB features with the help of thermal image features. Furthermore, with the benefits from thermal images, we rely only on a small temporal window to model the spatio-temporal information to greatly improve efficiency while maintaining detection accuracy. VT-VOD50 dataset consists of 50 pairs of challenging RGBT video sequences with complex backgrounds, various objects and different illuminations, which are collected in real traffic scenarios. Extensive experiments on VT-VOD50 dataset demonstrate the effectiveness and efficiency of our proposed method against existing mainstream VOD methods. The code of EINet and the dataset will be released to the public for free academic usage.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 66 references that are in the map. First posted on arXiv 3 Aug 2023.

Builds on

43 of 66 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  9. 2017Squeeze-and-Excitation Networkscited by 4,763
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2018YOLOv3: An Incremental Improvementcited by 2,652
  12. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
31 more
  1. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  2. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  3. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  4. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  5. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  6. 2018Path Aggregation Network for Instance Segmentationcited by 796
  7. 2017Focal Loss for Dense Object Detectioncited by 749
  8. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  9. 2015Recent Advances in Convolutional Neural Networkscited by 360
  10. 2016Deep Feature Flow for Video Recognitioncited by 186
  11. 2017Flow-Guided Feature Aggregation for Video Object Detectioncited by 177
  12. 2020Memory Enhanced Global-Local Aggregation for Video Object Detectioncited by 128
  13. 2018RGB-T Object Tracking:Benchmark and Baselinecited by 121
  14. 2020Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinementcited by 101
  15. 2022Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baselinecited by 90
  16. 2019Sequence Level Semantics Aggregation for Video Object Detectioncited by 87
  17. 2022TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformerscited by 78
  18. 2018Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detectioncited by 74
  19. 2020RGBT Salient Object Detection: A Large-scale Dataset and Benchmarkcited by 69
  20. 2016Seq-NMS for Video Object Detectioncited by 68
  21. 2019RGB-T Image Saliency Detection via Collaborative Graph Learningcited by 58
  22. 2020Challenge-Aware RGBT Trackingcited by 53
  23. 2017A Unified RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and A Novel Approachcited by 51
  24. 2019Context R-CNN: Long Term Temporal Context for Per-Camera Object Detectioncited by 45
  25. 2020Multi-interactive Dual-decoder for RGB-thermal Salient Object Detectioncited by 44
  26. 2021Temporal RoI Align for Video Object Recognitioncited by 44
  27. 2021End-to-End Video Object Detection with Spatial-Temporal Transformerscited by 40
  28. 2017Learning Cross-Modal Deep Representations for Robust Pedestrian Detectioncited by 34
  29. 2021Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separationcited by 32
  30. 2022BoxMask: Revisiting Bounding Box Supervision for Video Object Detectioncited by 6
  31. 2018The Intelligent ICU Pilot Study: Using Artificial Intelligence Technology for Autonomous Patient Monitoringcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Trackingcited by 43
  2. 2024Mixture of Scale Experts for Alignment-free RGBT Video Object Detection and A Unified Benchmarkcited by 4
  3. 2025Multimodal Spatio-temporal Graph Learning for Alignment-free RGBT Video Object Detectioncited by 0