mlsanityData to 7 Oct 2026

Investigating the Robustness and Properties of Detection Transformers (DETR) Toward Difficult Images

arXiv 2310.08772 ↗First posted 12 Oct 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Transformer-based object detectors (DETR) have shown significant performance across machine vision tasks, ultimately in object detection. This detector is based on a self-attention mechanism along with the transformer encoder-decoder architecture to capture the global context in the image. The critical issue to be addressed is how this model architecture can handle different image nuisances, such as occlusion and adversarial perturbations. We studied this issue by measuring the performance of DETR with different experiments and benchmarking the network with convolutional neural network (CNN) based detectors like YOLO and Faster-RCNN. We found that DETR performs well when it comes to resistance to interference from information loss in occlusion images. Despite that, we found that the adversarial stickers put on the image require the network to produce a new unnecessary set of keys, queries, and values, which in most cases, results in a misdirection of the network. DETR also performed poorer than YOLOv5 in the image corruption benchmark. Furthermore, we found that DETR depends heavily on the main query when making a prediction, which leads to imbalanced contributions between queries since the main query receives most of the gradient flow.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 18 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

17 of 18 references.

References · 17

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  6. 2021Intriguing Properties of Vision Transformerscited by 446
  7. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  8. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  9. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  10. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  11. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  12. 2017Robust Physical-World Attacks on Deep Learning Modelscited by 70
5 more
  1. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  2. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  3. 2022Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?cited by 47
  4. 2021Reveal of Vision Transformers Robustness against Adversarial Attackscited by 43
  5. 2021Occlusion Handling in Generic Object Detection: A Reviewcited by 25

Built on by

Not cited yet.