# MvCo-DoT:Multi-View Contrastive Domain Transfer Network for Medical Report Generation

arXiv: https://arxiv.org/abs/2304.07465 · HTML: https://mlsanity.com/paper/2304.07465/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 15 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [X-Linear Attention Networks for Image Captioning](https://mlsanity.com/paper/2003.14080/) (2020)
- [Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation](https://mlsanity.com/paper/2106.06963/) (2021)
- [RSG: A Simple but Effective Module for Learning Imbalanced Datasets](https://mlsanity.com/paper/2106.09859/) (2021)
- [MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation](https://mlsanity.com/paper/2102.10663/) (2021)
- [Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation](https://mlsanity.com/paper/2109.12242/) (2021)
- [RATCHET: Medical Transformer for Chest X-ray Diagnosis and Reporting](https://mlsanity.com/paper/2107.02104/) (2021)
- [Hybrid Reinforced Medical Report Generation with M-Linear Attention and Repetition Penalty](https://mlsanity.com/paper/2210.13729/) (2022)

## Built on by (most cited first, 2 of 2)

- [Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models](https://mlsanity.com/paper/2506.08990/) (2025)
- [Med-gte-hybrid: A contextual embedding transformer model for extracting actionable information from clinical texts](https://mlsanity.com/paper/2502.15996/) (2025)

## Abstract (arXiv)

In clinical scenarios, multiple medical images with different views are usually generated at the same time, and they have high semantic consistency. However, the existing medical report generation methods cannot exploit the rich multi-view mutual information of medical images. Therefore, in this work, we propose the first multi-view medical report generation model, called MvCo-DoT. Specifically, MvCo-DoT first propose a multi-view contrastive learning (MvCo) strategy to help the deep reinforcement learning based model utilize the consistency of multi-view inputs for better model learning. Then, to close the performance gaps of using multi-view and single-view inputs, a domain transfer network is further proposed to ensure MvCo-DoT achieve almost the same performance as multi-view inputs using only single-view inputs.Extensive experiments on the IU X-Ray public dataset show that MvCo-DoT outperforms the SOTA medical report generation baselines in all metrics.