# Ultrasound Report Generation with Cross-Modality Feature Alignment via Unsupervised Guidance

arXiv: https://arxiv.org/abs/2406.00644 · HTML: https://mlsanity.com/paper/2406.00644/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 2 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [Exploring Visual Relationship for Image Captioning](https://mlsanity.com/paper/1809.07041/) (2018)
- [When Radiology Report Generation Meets Knowledge Graph](https://mlsanity.com/paper/2002.08277/) (2020)
- [Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation](https://mlsanity.com/paper/1903.10122/) (2019)
- [Knowledge Matters: Radiology Report Generation with General and Specific Knowledge](https://mlsanity.com/paper/2112.15009/) (2021)
- [Human-like Controllable Image Captioning with Verb-specific Semantic Roles](https://mlsanity.com/paper/2103.12204/) (2021)
- [Reflective Decoding Network for Image Captioning](https://mlsanity.com/paper/1908.11824/) (2019)
- [Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives](https://mlsanity.com/paper/2307.05545/) (2023)
- [Towards Accurate Text-based Image Captioning with Content Diversity Exploration](https://mlsanity.com/paper/2105.03236/) (2021)
- [Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning](https://mlsanity.com/paper/2108.05067/) (2021)
- [A Self-Guided Framework for Radiology Report Generation](https://mlsanity.com/paper/2206.09378/) (2022)
- [DeltaNet:Conditional Medical Report Generation for COVID-19 Diagnosis](https://mlsanity.com/paper/2211.13229/) (2022)

## Built on by (most cited first, 17 of 17)

- [U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding](https://mlsanity.com/paper/2505.17779/) (2025)
- [A Fully Open and Generalizable Foundation Model for Ultrasound Clinical Applications](https://mlsanity.com/paper/2509.11752/) (2025)
- [Speckle2Self: Self-Supervised Ultrasound Speckle Reduction Without Clean Data](https://mlsanity.com/paper/2507.06828/) (2025)
- [Adapting Vision-Language Foundation Model for Next Generation Medical Ultrasound Image Analysis](https://mlsanity.com/paper/2506.08849/) (2025)
- [EchoVLM: Dynamic Mixture-of-Experts Vision-Language Model for Universal Ultrasound Intelligence](https://mlsanity.com/paper/2509.14977/) (2025)
- [Enhancing Abnormality Grounding for Vision Language Models with Knowledge Descriptions](https://mlsanity.com/paper/2503.03278/) (2025)
- [Ultrasound Report Generation with Multimodal Large Language Models for Standardized Texts](https://mlsanity.com/paper/2505.08838/) (2025)
- [Knowledge to Sight: Reasoning over Visual Attributes via Knowledge Decomposition for Abnormality Grounding](https://mlsanity.com/paper/2508.04572/) (2025)
- [COME: Dual Structure-Semantic Learning with Collaborative MoE for Universal Lesion Detection Across Heterogeneous Ultrasound Datasets](https://mlsanity.com/paper/2508.09886/) (2025)
- [HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Tasks](https://mlsanity.com/paper/2508.12778/) (2025)
- [A Vision-language Framework for Comparative Reasoning in Radiology](https://mlsanity.com/paper/2606.06407/) (2026)
- [Uterine Ultrasound Image Captioning Using Deep Learning Techniques](https://mlsanity.com/paper/2411.14039/) (2024)
- [BUSTR: Descriptor-Aware Vision-Language Learning for Breast Ultrasound Report Generation](https://mlsanity.com/paper/2511.20956/) (2025)
- [Ultrasound Vision-Language Alignment via Contrastive Learning](https://mlsanity.com/paper/2605.02126/) (2026)
- [DECIS: Dual-Evidence Corrective Verification for Interpretable Strabismus Diagnostic Decision-Making](https://mlsanity.com/paper/2606.09249/) (2026)
- [STAIL: Semantic Text-Anchored Incremental Learning for Medical Imaging via Large Language Models](https://mlsanity.com/paper/2608.05808/) (2026)
- [A Concept-informed Interpretable Artificial Intelligence Framework for Breast Ultrasound Diagnosis](https://mlsanity.com/paper/2610.16128/) (2026)

## Abstract (arXiv)

Automatic report generation has arisen as a significant research area in computer-aided diagnosis, aiming to alleviate the burden on clinicians by generating reports automatically based on medical images. In this work, we propose a novel framework for automatic ultrasound report generation, leveraging a combination of unsupervised and supervised learning methods to aid the report generation process. Our framework incorporates unsupervised learning methods to extract potential knowledge from ultrasound text reports, serving as the prior information to guide the model in aligning visual and textual features, thereby addressing the challenge of feature discrepancy. Additionally, we design a global semantic comparison mechanism to enhance the performance of generating more comprehensive and accurate medical reports. To enable the implementation of ultrasound report generation, we constructed three large-scale ultrasound image-text datasets from different organs for training and validation purposes. Extensive evaluations with other state-of-the-art approaches exhibit its superior performance across all three datasets. Code and dataset are valuable at this link.