# Taming Detection Transformers for Medical Object Detection

arXiv: https://arxiv.org/abs/2306.15472 · HTML: https://mlsanity.com/paper/2306.15472/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 10 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 7 of 7)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [nnDetection: A Self-configuring Method for Medical Object Detection](https://mlsanity.com/paper/2106.00817/) (2021)
- [Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection](https://mlsanity.com/paper/1811.08661/) (2018)
- [Focused Decoding Enables 3D Anatomical Detection by Transformers](https://mlsanity.com/paper/2207.10774/) (2022)

## Built on by (most cited first, 4 of 4)

- [CLIP in Medical Imaging: A Survey](https://mlsanity.com/paper/2312.07353/) (2023)
- [Arena: A Patch-of-Interest ViT Inference Acceleration System for Edge-Assisted Video Analytics](https://mlsanity.com/paper/2404.09245/) (2024)
- [Heatmap Guided Query Transformers for Robust Astrocyte Detection across Immunostains and Resolutions](https://mlsanity.com/paper/2509.03323/) (2025)
- [Align Your Query: Representation Alignment for Multimodality Medical Object Detection](https://mlsanity.com/paper/2510.02789/) (2025)

## Abstract (arXiv)

The accurate detection of suspicious regions in medical images is an error-prone and time-consuming process required by many routinely performed diagnostic procedures. To support clinicians during this difficult task, several automated solutions were proposed relying on complex methods with many hyperparameters. In this study, we investigate the feasibility of DEtection TRansformer (DETR) models for volumetric medical object detection. In contrast to previous works, these models directly predict a set of objects without relying on the design of anchors or manual heuristics such as non-maximum-suppression to detect objects. We show by conducting extensive experiments with three models, namely DETR, Conditional DETR, and DINO DETR on four data sets (CADA, RibFrac, KiTS19, and LIDC) that these set prediction models can perform on par with or even better than currently existing methods. DINO DETR, the best-performing model in our experiments demonstrates this by outperforming a strong anchor-based one-stage detector, Retina U-Net, on three out of four data sets.