# EchoNarrator: Generating natural text explanations for ejection fraction predictions

arXiv: https://arxiv.org/abs/2410.23744 · HTML: https://mlsanity.com/paper/2410.23744/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [Generating Visual Explanations](https://mlsanity.com/paper/1603.08507/) (2016)
- [Grounding Visual Explanations](https://mlsanity.com/paper/1807.09685/) (2018)
- [NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks](https://mlsanity.com/paper/2203.05081/) (2022)
- [Ultrasound Video Transformers for Cardiac Ejection Fraction Estimation](https://mlsanity.com/paper/2107.00977/) (2021)
- [Explaining Chest X-ray Pathologies in Natural Language](https://mlsanity.com/paper/2207.04343/) (2022)
- [EchoGNN: Explainable Ejection Fraction Estimation with Graph Neural Networks](https://mlsanity.com/paper/2208.14003/) (2022)
- [EchoCoTr: Estimation of the Left Ventricular Ejection Fraction from Spatiotemporal Echocardiography](https://mlsanity.com/paper/2209.04242/) (2022)
- [Light-weight spatio-temporal graphs for segmentation and ejection fraction prediction in cardiac ultrasound](https://mlsanity.com/paper/2207.02549/) (2022)
- [Cyclical Self-Supervision for Semi-Supervised Ejection Fraction Prediction from Echocardiogram Videos](https://mlsanity.com/paper/2210.11291/) (2022)
- [Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks](https://mlsanity.com/paper/2308.09033/) (2023)
- [GEMTrans: A General, Echocardiography-based, Multi-Level Transformer Framework for Cardiovascular Diagnosis](https://mlsanity.com/paper/2308.13217/) (2023)

## Built on by (most cited first, 2 of 2)

- [Echo-E3Net: Efficient Endocardial Spatio-Temporal Network for Ejection Fraction Estimation](https://mlsanity.com/paper/2503.17543/) (2025)
- [Mamba-3D as Masked Autoencoders for Accurate and Data-Efficient Analysis of Medical Ultrasound Videos](https://mlsanity.com/paper/2503.20258/) (2025)

## Abstract (arXiv)

Ejection fraction (EF) of the left ventricle (LV) is considered as one of the most important measurements for diagnosing acute heart failure and can be estimated during cardiac ultrasound acquisition. While recent successes in deep learning research successfully estimate EF values, the proposed models often lack an explanation for the prediction. However, providing clear and intuitive explanations for clinical measurement predictions would increase the trust of cardiologists in these models. In this paper, we explore predicting EF measurements with Natural Language Explanation (NLE). We propose a model that in a single forward pass combines estimation of the LV contour over multiple frames, together with a set of modules and routines for computing various motion and shape attributes that are associated with ejection fraction. It then feeds the attributes into a large language model to generate text that helps to explain the network's outcome in a human-like manner. We provide experimental evaluation of our explanatory output, as well as EF prediction, and show that our model can provide EF comparable to state-of-the-art together with meaningful and accurate natural language explanation to the prediction. The project page can be found at https://github.com/guybenyosef/EchoNarrator .