# MvKeTR: Chest CT Report Generation with Multi-View Perception and Knowledge Enhancement

arXiv: https://arxiv.org/abs/2411.18309 · HTML: https://mlsanity.com/paper/2411.18309/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 66 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [Meshed-Memory Transformer for Image Captioning](https://mlsanity.com/paper/1912.08226/) (2019)
- [Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning](https://mlsanity.com/paper/1612.01887/) (2016)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [Cross-modal Memory Networks for Radiology Report Generation](https://mlsanity.com/paper/2204.13258/) (2022)
- [CT2Rep: Automated Radiology Report Generation for 3D Medical Imaging](https://mlsanity.com/paper/2403.06801/) (2024)
- [Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment](https://mlsanity.com/paper/2112.15011/) (2021)
- [Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes](https://mlsanity.com/paper/2002.04752/) (2020)
- [GenerateCT: Text-Conditional Generation of 3D Chest CT Volumes](https://mlsanity.com/paper/2305.16037/) (2023)
- [KiUT: Knowledge-injected U-Transformer for Radiology Report Generation](https://mlsanity.com/paper/2306.11345/) (2023)
- [Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment](https://mlsanity.com/paper/1907.09085/) (2019)
- [CPTR: Full Transformer Network for Image Captioning](https://mlsanity.com/paper/2101.10804/) (2021)
- [Progressive Transformer-Based Generation of Radiology Reports](https://mlsanity.com/paper/2102.09777/) (2021)
- [On the expressiveness and spectral bias of KANs](https://mlsanity.com/paper/2410.01803/) (2024)
- [Dia-LLaMA: Towards Large Language Model-driven CT Report Generation](https://mlsanity.com/paper/2403.16386/) (2024)
- [Large Language Model with Region-guided Referring and Grounding for CT Report Generation](https://mlsanity.com/paper/2411.15539/) (2024)
- [CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generation](https://mlsanity.com/paper/2211.01412/) (2022)

## Built on by (most cited first, 4 of 4)

- [DiffVP: Differential Visual Semantic Prompting for LLM-Based CT Report Generation](https://mlsanity.com/paper/2603.17718/) (2026)
- [SliceWorld: A Predictive and Controllable World-State Model for CT Report Generation](https://mlsanity.com/paper/2605.24371/) (2026)
- [MonteRET: AI Agent Enhancing Multimodal LLMs with Multi-granularity Knowledge Retrieval for Chest CT Report Generation](https://mlsanity.com/paper/2607.14264/) (2026)
- [Unleashing Video Language Models for Fine-grained HRCT Report Generation](https://mlsanity.com/paper/2603.12469/) (2026)

## Abstract (arXiv)

CT report generation (CTRG) aims to automatically generate diagnostic reports for 3D volumes, relieving clinicians' workload and improving patient care. Despite clinical value, existing works fail to effectively incorporate diagnostic information from multiple anatomical views and lack related clinical expertise essential for accurate and reliable diagnosis. To resolve these limitations, we propose a novel Multi-view perception Knowledge-enhanced TansfoRmer (MvKeTR) to mimic the diagnostic workflow of clinicians. Just as radiologists first examine CT scans from multiple planes, a Multi-View Perception Aggregator (MVPA) with view-aware attention is proposed to synthesize diagnostic information from multiple anatomical views effectively. Then, inspired by how radiologists further refer to relevant clinical records to guide diagnostic decision-making, a Cross-Modal Knowledge Enhancer (CMKE) is devised to retrieve the most similar reports based on the query volume to incorporate domain knowledge into the diagnosis procedure. Furthermore, instead of traditional MLPs, we employ Kolmogorov-Arnold Networks (KANs) as the fundamental building blocks of both modules, which exhibit superior parameter efficiency and reduced spectral bias to better capture high-frequency components critical for CT interpretation while mitigating overfitting. Extensive experiments on the public CTRG-Chest-548 K dataset demonstrate that our method outpaces prior state-of-the-art (SOTA) models across almost all metrics. The code is available at https://github.com/xiweideng/MvKeTR.