# From Slices to Sequences: Autoregressive Tracking Transformer for Cohesive and Consistent 3D Lymph Node Detection in CT Scans

arXiv: https://arxiv.org/abs/2503.07933 · HTML: https://mlsanity.com/paper/2503.07933/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 29 of 29)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)
- [VITA: Video Instance Segmentation via Object Token Association](https://mlsanity.com/paper/2206.04403/) (2022)
- [nnDetection: A Self-configuring Method for Medical Object Detection](https://mlsanity.com/paper/2106.00817/) (2021)
- [A Generalized Framework for Video Instance Segmentation](https://mlsanity.com/paper/2211.08834/) (2022)
- [FocalMix: Semi-Supervised Learning for 3D Medical Image Detection](https://mlsanity.com/paper/2003.09108/) (2020)
- [Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection](https://mlsanity.com/paper/1811.08661/) (2018)
- [Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT](https://mlsanity.com/paper/2009.02577/) (2020)
- [MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentation](https://mlsanity.com/paper/1908.04373/) (2019)
- [Mediastinal lymph nodes segmentation using 3D convolutional neural network ensembles and anatomical priors guiding](https://mlsanity.com/paper/2102.06515/) (2021)
- [SATr: Slice Attention with Transformer for Universal Lesion Detection](https://mlsanity.com/paper/2203.07373/) (2022)
- [Lymph Node Gross Tumor Volume Detection and Segmentation via Distance-based Gating using 3D CT/PET Imaging in Radiotherapy](https://mlsanity.com/paper/2008.11870/) (2020)
- [Asymmetric 3D Context Fusion for Universal Lesion Detection](https://mlsanity.com/paper/2109.08684/) (2021)
- [Fracture Detection in Pediatric Wrist Trauma X-ray Images Using YOLOv8 Algorithm](https://mlsanity.com/paper/2304.05071/) (2023)
- [DeepStationing: Thoracic Lymph Node Station Parsing in CT Scans using Anatomical Context Encoding and Key Organ Auto-Search](https://mlsanity.com/paper/2109.09271/) (2021)
- [Detecting Scatteredly-Distributed, Small, andCritically Important Objects in 3D OncologyImaging via Decision Stratification](https://mlsanity.com/paper/2005.13705/) (2020)
- [Anatomy-Aware Lymph Node Detection in Chest CT using Implicit Station Stratification](https://mlsanity.com/paper/2307.15271/) (2023)
- [Advancing 3D Medical Image Analysis with Variable Dimension Transform based Supervised 3D Pre-training](https://mlsanity.com/paper/2201.01426/) (2022)
- [DiffULD: Diffusive Universal Lesion Detection](https://mlsanity.com/paper/2303.15728/) (2023)
- [Effective Lymph Nodes Detection in CT Scans Using Location Debiased Query Selection and Contrastive Query Representation in Transformer](https://mlsanity.com/paper/2404.03819/) (2024)

## Abstract (arXiv)

Lymph node (LN) assessment is an essential task in the routine radiology workflow, providing valuable insights for cancer staging, treatment planning and beyond. Identifying scatteredly-distributed and low-contrast LNs in 3D CT scans is highly challenging, even for experienced clinicians. Previous lesion and LN detection methods demonstrate effectiveness of 2.5D approaches (i.e, using 2D network with multi-slice inputs), leveraging pretrained 2D model weights and showing improved accuracy as compared to separate 2D or 3D detectors. However, slice-based 2.5D detectors do not explicitly model inter-slice consistency for LN as a 3D object, requiring heuristic post-merging steps to generate final 3D LN instances, which can involve tuning a set of parameters for each dataset. In this work, we formulate 3D LN detection as a tracking task and propose LN-Tracker, a novel LN tracking transformer, for joint end-to-end detection and 3D instance association. Built upon DETR-based detector, LN-Tracker decouples transformer decoder's query into the track and detection groups, where the track query autoregressively follows previously tracked LN instances along the z-axis of a CT scan. We design a new transformer decoder with masked attention module to align track query's content to the context of current slice, meanwhile preserving detection query's high accuracy in current slice. An inter-slice similarity loss is introduced to encourage cohesive LN association between slices. Extensive evaluation on four lymph node datasets shows LN-Tracker's superior performance, with at least 2.7% gain in average sensitivity when compared to other top 3D/2.5D detectors. Further validation on public lung nodule and prostate tumor detection tasks confirms the generalizability of LN-Tracker as it achieves top performance on both tasks.