# Accurate and Robust Lesion RECIST Diameter Prediction and Segmentation with Transformers

arXiv: https://arxiv.org/abs/2208.13113 · HTML: https://mlsanity.com/paper/2208.13113/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 28 Aug 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [Medical Image Segmentation Using Squeeze-and-Expansion Transformers](https://mlsanity.com/paper/2105.09511/) (2021)
- [Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies](https://mlsanity.com/paper/2012.04872/) (2020)
- [Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST](https://mlsanity.com/paper/1807.01172/) (2018)
- [Weakly-Supervised Universal Lesion Segmentation with Regional Level Set Loss](https://mlsanity.com/paper/2105.01218/) (2021)
- [One Click Lesion RECIST Measurement and Segmentation on CT Scans](https://mlsanity.com/paper/2007.11087/) (2020)
- [Lesion Segmentation and RECIST Diameter Prediction via Click-driven Attention and Dual-path Connection](https://mlsanity.com/paper/2105.01828/) (2021)
- [Weakly Supervised Lesion Co-segmentation on CT Scans](https://mlsanity.com/paper/2001.09174/) (2020)
- [Semi-Automatic RECIST Labeling on CT Scans with Cascaded Convolutional Neural Networks](https://mlsanity.com/paper/1806.09507/) (2018)
- [CT Image Enhancement Using Stacked Generative Adversarial Networks and Transfer Learning for Lesion Segmentation Improvement](https://mlsanity.com/paper/1807.07144/) (2018)
- [Weakly-Supervised Lesion Segmentation on CT Scans using Co-Segmentation](https://mlsanity.com/paper/2001.08590/) (2020)

## Built on by (most cited first, 4 of 4)

- [The ULS23 Challenge: a Baseline Model and Benchmark Dataset for 3D Universal Lesion Segmentation in Computed Tomography](https://mlsanity.com/paper/2406.05231/) (2024)
- [Towards Automated Polyp Segmentation Using Weakly- and Semi-Supervised Learning and Deformable Transformers](https://mlsanity.com/paper/2211.11847/) (2022)
- [Scale-aware Test-time Click Adaptation for Pulmonary Nodule and Mass Segmentation](https://mlsanity.com/paper/2307.15645/) (2023)
- [Learning Nuclei Representations with Masked Image Modelling](https://mlsanity.com/paper/2306.17116/) (2023)

## Abstract (arXiv)

Automatically measuring lesion/tumor size with RECIST (Response Evaluation Criteria In Solid Tumors) diameters and segmentation is important for computer-aided diagnosis. Although it has been studied in recent years, there is still space to improve its accuracy and robustness, such as (1) enhancing features by incorporating rich contextual information while keeping a high spatial resolution and (2) involving new tasks and losses for joint optimization. To reach this goal, this paper proposes a transformer-based network (MeaFormer, Measurement transFormer) for lesion RECIST diameter prediction and segmentation (LRDPS). It is formulated as three correlative and complementary tasks: lesion segmentation, heatmap prediction, and keypoint regression. To the best of our knowledge, it is the first time to use keypoint regression for RECIST diameter prediction. MeaFormer can enhance high-resolution features by employing transformers to capture their long-range dependencies. Two consistency losses are introduced to explicitly build relationships among these tasks for better optimization. Experiments show that MeaFormer achieves the state-of-the-art performance of LRDPS on the large-scale DeepLesion dataset and produces promising results of two downstream clinic-relevant tasks, i.e., 3D lesion segmentation and RECIST assessment in longitudinal studies.