# Focused Decoding Enables 3D Anatomical Detection by Transformers

arXiv: https://arxiv.org/abs/2207.10774 · HTML: https://mlsanity.com/paper/2207.10774/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Efficient DETR: Improving End-to-End Object Detector with Dense Prior](https://mlsanity.com/paper/2104.01318/) (2021)
- [Relationformer: A Unified Framework for Image-to-Graph Generation](https://mlsanity.com/paper/2203.10202/) (2022)
- [nnDetection: A Self-configuring Method for Medical Object Detection](https://mlsanity.com/paper/2106.00817/) (2021)
- [Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection](https://mlsanity.com/paper/1811.08661/) (2018)
- [Attention-Based Transformers for Instance Segmentation of Cells in Microstructures](https://mlsanity.com/paper/2011.09763/) (2020)
- [COTR: Convolution in Transformer Network for End to End Polyp Detection](https://mlsanity.com/paper/2105.10925/) (2021)
- [Shape-Aware Complementary-Task Learning for Multi-Organ Segmentation](https://mlsanity.com/paper/1908.05099/) (2019)
- [Evaluating the Robustness of Self-Supervised Learning in Medical Imaging](https://mlsanity.com/paper/2105.06986/) (2021)
- [A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning](https://mlsanity.com/paper/2203.00624/) (2022)

## Built on by (most cited first, 6 of 6)

- [Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review](https://mlsanity.com/paper/2301.03505/) (2023)
- [CLIP in Medical Imaging: A Survey](https://mlsanity.com/paper/2312.07353/) (2023)
- [DeViDe: Faceted medical knowledge for improved medical vision-language pre-training](https://mlsanity.com/paper/2404.03618/) (2024)
- [Taming Detection Transformers for Medical Object Detection](https://mlsanity.com/paper/2306.15472/) (2023)
- [BifDet: A 3D Bifurcation Detection Dataset for Airway-Tree Modeling](https://mlsanity.com/paper/2604.24999/) (2026)
- [YOLO-PVC: 2D-to-3D Consolidation of Slice-wise Detections for Volumetric Liver Tumor Localization in MRI](https://mlsanity.com/paper/2608.04642/) (2026)

## Abstract (arXiv)

Detection Transformers represent end-to-end object detection approaches based on a Transformer encoder-decoder architecture, exploiting the attention mechanism for global relation modeling. Although Detection Transformers deliver results on par with or even superior to their highly optimized CNN-based counterparts operating on 2D natural images, their success is closely coupled to access to a vast amount of training data. This, however, restricts the feasibility of employing Detection Transformers in the medical domain, as access to annotated data is typically limited. To tackle this issue and facilitate the advent of medical Detection Transformers, we propose a novel Detection Transformer for 3D anatomical structure detection, dubbed Focused Decoder. Focused Decoder leverages information from an anatomical region atlas to simultaneously deploy query anchors and restrict the cross-attention's field of view to regions of interest, which allows for a precise focus on relevant anatomical structures. We evaluate our proposed approach on two publicly available CT datasets and demonstrate that Focused Decoder not only provides strong detection results and thus alleviates the need for a vast amount of annotated data but also exhibits exceptional and highly intuitive explainability of results via attention weights. Our code is available at https://github.com/bwittmann/transoar.