# X-ray illicit object detection using hybrid CNN-transformer neural network architectures

arXiv: https://arxiv.org/abs/2505.00564 · HTML: https://mlsanity.com/paper/2505.00564/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 1 May 2025.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [TinyViT: Fast Pretraining Distillation for Small Vision Transformers](https://mlsanity.com/paper/2207.10666/) (2022)
- [Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity](https://mlsanity.com/paper/2111.14330/) (2021)
- [Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios](https://mlsanity.com/paper/2207.05501/) (2022)
- [Occluded Prohibited Items Detection: an X-ray Security Inspection Benchmark and De-occlusion Attention Module](https://mlsanity.com/paper/2004.08656/) (2020)
- [A survey of the Vision Transformers and their CNN-Transformer based Variants](https://mlsanity.com/paper/2305.09880/) (2023)
- [Towards Real-World Prohibited Item Detection: A Large-Scale X-ray Benchmark](https://mlsanity.com/paper/2108.07020/) (2021)
- [Towards Real-world X-ray Security Inspection: A High-Quality Benchmark and Lateral Inhibition Module for Prohibited Items Detection](https://mlsanity.com/paper/2108.09917/) (2021)
- [PIDray: A Large-scale X-ray Benchmark for Real-World Prohibited Item Detection](https://mlsanity.com/paper/2211.10763/) (2022)
- [Self-supervised visual learning in the low-data regime: a comparative evaluation](https://mlsanity.com/paper/2404.17202/) (2024)
- [Computer Vision on X-ray Data in Industrial Production and Security Applications: A Comprehensive Survey](https://mlsanity.com/paper/2211.05565/) (2022)
- [Illicit item detection in X-ray images for security applications](https://mlsanity.com/paper/2305.01936/) (2023)
- [Visual inspection for illicit items in X-ray images using Deep Learning](https://mlsanity.com/paper/2310.03658/) (2023)
- [Illicit object detection in X-ray images using Vision Transformers](https://mlsanity.com/paper/2403.19043/) (2024)

## Abstract (arXiv)

In the field of X-ray security applications, even the smallest details can significantly impact outcomes. Objects that are heavily occluded or intentionally concealed pose a great challenge for detection, whether by human observation or through advanced technological applications. While certain Deep Learning (DL) architectures demonstrate strong performance in processing local information, such as Convolutional Neural Networks (CNNs), others excel in handling distant information, e.g., transformers. In X-ray security imaging the literature has been dominated by the use of CNN-based methods, while the integration of the two aforementioned leading architectures has not been sufficiently explored. In this paper, various hybrid CNN-transformer architectures are evaluated against a common CNN object detection baseline, namely YOLOv8. In particular, a CNN (HGNetV2) and a hybrid CNN-transformer (Next-ViT-S) backbone are combined with different CNN/transformer detection heads (YOLOv8 and RT-DETR). The resulting architectures are comparatively evaluated on three challenging public X-ray inspection datasets, namely EDS, HiXray, and PIDray. Interestingly, while the YOLOv8 detector with its default backbone (CSP-DarkNet53) is generally shown to be advantageous on the HiXray and PIDray datasets, when a domain distribution shift is incorporated in the X-ray images (as happens in the EDS datasets), hybrid CNN-transformer architectures exhibit increased robustness. Detailed comparative evaluation results, including object-level detection performance and object-size error analysis, demonstrate the strengths and weaknesses of each architectural combination and suggest guidelines for future research. The source code and network weights of the models employed in this study are available at https://github.com/jgenc/xray-comparative-evaluation.