# MUVF-YOLOX: A Multi-modal Ultrasound Video Fusion Network for Renal Tumor Diagnosis

arXiv: https://arxiv.org/abs/2307.07807 · HTML: https://mlsanity.com/paper/2307.07807/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 15 Jul 2023.

## Builds on (most cited first, 9 of 9)

- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [Deep Multimodal Fusion by Channel Exchanging](https://mlsanity.com/paper/2011.05005/) (2020)
- [A New Dataset and A Baseline Model for Breast Lesion Detection in Ultrasound Videos](https://mlsanity.com/paper/2207.00141/) (2022)
- [YOLOV: Making Still Image Object Detectors Great at Video Object Detection](https://mlsanity.com/paper/2208.09686/) (2022)
- [Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction](https://mlsanity.com/paper/2112.02252/) (2021)
- [Personalized Diagnostic Tool for Thyroid Cancer Classification using Multi-view Ultrasound](https://mlsanity.com/paper/2207.00496/) (2022)
- [TranSiam: Fusing Multimodal Visual Features Using Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2204.12185/) (2022)
- [Weighted Concordance Index Loss-based Multimodal Survival Modeling for Radiation Encephalopathy Assessment in Nasopharyngeal Carcinoma Radiotherapy](https://mlsanity.com/paper/2206.11458/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Spatial-Temporal Progressive Fusion Network for Breast Lesion Segmentation in Ultrasound Videos](https://mlsanity.com/paper/2403.11699/) (2024)
- [TASL-Net: Tri-Attention Selective Learning Network for Intelligent Diagnosis of Bimodal Ultrasound Video](https://mlsanity.com/paper/2409.01557/) (2024)

## Abstract (arXiv)

Early diagnosis of renal cancer can greatly improve the survival rate of patients. Contrast-enhanced ultrasound (CEUS) is a cost-effective and non-invasive imaging technique and has become more and more frequently used for renal tumor diagnosis. However, the classification of benign and malignant renal tumors can still be very challenging due to the highly heterogeneous appearance of cancer and imaging artifacts. Our aim is to detect and classify renal tumors by integrating B-mode and CEUS-mode ultrasound videos. To this end, we propose a novel multi-modal ultrasound video fusion network that can effectively perform multi-modal feature fusion and video classification for renal tumor diagnosis. The attention-based multi-modal fusion module uses cross-attention and self-attention to extract modality-invariant features and modality-specific features in parallel. In addition, we design an object-level temporal aggregation (OTA) module that can automatically filter low-quality features and efficiently integrate temporal information from multiple frames to improve the accuracy of tumor diagnosis. Experimental results on a multicenter dataset show that the proposed framework outperforms the single-modal models and the competing methods. Furthermore, our OTA module achieves higher classification accuracy than the frame-level predictions. Our code is available at urlhttps://github.com/JeunyuLi/MUAF.