# RATCHET: Medical Transformer for Chest X-ray Diagnosis and Reporting

arXiv: https://arxiv.org/abs/2107.02104 · HTML: https://mlsanity.com/paper/2107.02104/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rays](https://mlsanity.com/paper/1801.04334/) (2018)
- [Clinically Accurate Chest X-Ray Report Generation](https://mlsanity.com/paper/1904.02633/) (2019)
- [Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation](https://mlsanity.com/paper/1903.10122/) (2019)
- [NegBio: a high-performance tool for negation and uncertainty detection in radiology reports](https://mlsanity.com/paper/1712.05898/) (2017)
- [Multi-domain Clinical Natural Language Processing with MedCAT: the Medical Concept Annotation Toolkit](https://mlsanity.com/paper/2010.01165/) (2020)
- [Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment](https://mlsanity.com/paper/1907.09085/) (2019)
- [Chest X-ray Report Generation through Fine-Grained Label Learning](https://mlsanity.com/paper/2007.13831/) (2020)

## Built on by (most cited first, 30 of 34)

- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review](https://mlsanity.com/paper/2301.03505/) (2023)
- [KiUT: Knowledge-injected U-Transformer for Radiology Report Generation](https://mlsanity.com/paper/2306.11345/) (2023)
- [Cross-modal Prototype Driven Network for Radiology Report Generation](https://mlsanity.com/paper/2207.04818/) (2022)
- [CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset](https://mlsanity.com/paper/2410.00379/) (2024)
- [Explaining Chest X-ray Pathologies in Natural Language](https://mlsanity.com/paper/2207.04343/) (2022)
- [Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding](https://mlsanity.com/paper/2210.03416/) (2022)
- [CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generation](https://mlsanity.com/paper/2211.01412/) (2022)
- [The Impact of Imperfect XAI on Human-AI Decision-Making](https://mlsanity.com/paper/2307.13566/) (2023)
- [Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting](https://mlsanity.com/paper/2307.05766/) (2023)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [A survey on attention mechanisms for medical applications: are we moving towards better algorithms?](https://mlsanity.com/paper/2204.12406/) (2022)
- [A Systematic Review of Deep Learning-based Research on Radiology Report Generation](https://mlsanity.com/paper/2311.14199/) (2023)
- [Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training](https://mlsanity.com/paper/2307.07246/) (2023)
- [LLaVA Needs More Knowledge: Retrieval Augmented Natural Language Generation with Knowledge Graph for Explaining Thoracic Pathologies](https://mlsanity.com/paper/2410.04749/) (2024)
- [A Survey of Medical Vision-and-Language Applications and Their Techniques](https://mlsanity.com/paper/2411.12195/) (2024)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data](https://mlsanity.com/paper/2405.12833/) (2024)
- [FlexR: Few-shot Classification with Language Embeddings for Structured Reporting of Chest X-rays](https://mlsanity.com/paper/2203.15723/) (2022)
- [SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images](https://mlsanity.com/paper/2302.03861/) (2023)
- [Learning To Guide Human Decision Makers With Vision-Language Models](https://mlsanity.com/paper/2403.16501/) (2024)
- [On the Importance of Image Encoding in Automated Chest X-Ray Report Generation](https://mlsanity.com/paper/2211.13465/) (2022)
- [Prior-RadGraphFormer: A Prior-Knowledge-Enhanced Transformer for Generating Radiology Graphs from X-Rays](https://mlsanity.com/paper/2303.13818/) (2023)
- [MvCo-DoT:Multi-View Contrastive Domain Transfer Network for Medical Report Generation](https://mlsanity.com/paper/2304.07465/) (2023)
- [Quality Control for Radiology Report Generation Models via Auxiliary Auditing Components](https://mlsanity.com/paper/2407.21638/) (2024)
- [Recurrent Visual Feature Extraction and Stereo Attentions for CT Report Generation](https://mlsanity.com/paper/2506.19665/) (2025)
- [Graph-Augmented Topological Internalization with Dual-Stream Classifiers for Medical Report Generation](https://mlsanity.com/paper/2605.02376/) (2026)
- [Learning to Guide Human Experts via Personalized Large Language Models](https://mlsanity.com/paper/2308.06039/) (2023)
- [Dynamic Multi-Domain Knowledge Networks for Chest X-ray Report Generation](https://mlsanity.com/paper/2310.05119/) (2023)
- [C^2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network](https://mlsanity.com/paper/2310.05355/) (2023)

## Abstract (arXiv)

Chest radiographs are one of the most common diagnostic modalities in clinical routine. It can be done cheaply, requires minimal equipment, and the image can be diagnosed by every radiologists. However, the number of chest radiographs obtained on a daily basis can easily overwhelm the available clinical capacities. We propose RATCHET: RAdiological Text Captioning for Human Examined Thoraces. RATCHET is a CNN-RNN-based medical transformer that is trained end-to-end. It is capable of extracting image features from chest radiographs, and generates medically accurate text reports that fit seamlessly into clinical work flows. The model is evaluated for its natural language generation ability using common metrics from NLP literature, as well as its medically accuracy through a surrogate report classification task. The model is available for download at: http://www.github.com/farrell236/RATCHET.