# FFA Sora, video generation as fundus fluorescein angiography simulator

arXiv: https://arxiv.org/abs/2412.17346 · HTML: https://mlsanity.com/paper/2412.17346/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [Open-Sora Plan: Open-Source Large Video Generation Model](https://mlsanity.com/paper/2412.00131/) (2024)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis](https://mlsanity.com/paper/2305.18453/) (2023)
- [WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Model](https://mlsanity.com/paper/2411.17459/) (2024)
- [Endora: Video Generation Models as Endoscopy Simulators](https://mlsanity.com/paper/2403.11050/) (2024)
- [EyeDiff: text-to-image diffusion model improves rare eye disease diagnosis](https://mlsanity.com/paper/2411.10004/) (2024)
- [Fundus2Video: Cross-Modal Angiography Video Generation from Static Fundus Photography with Clinical Knowledge Guidance](https://mlsanity.com/paper/2408.15217/) (2024)
- [Fundus to Fluorescein Angiography Video Generation as a Retinal Generative Foundation Model](https://mlsanity.com/paper/2410.13242/) (2024)

## Built on by (most cited first, 1 of 1)

- [AI-powered virtual eye: perspective, challenges and opportunities](https://mlsanity.com/paper/2505.05516/) (2025)

## Abstract (arXiv)

Fundus fluorescein angiography (FFA) is critical for diagnosing retinal vascular diseases, but beginners often struggle with image interpretation. This study develops FFA Sora, a text-to-video model that converts FFA reports into dynamic videos via a Wavelet-Flow Variational Autoencoder (WF-VAE) and a diffusion transformer (DiT). Trained on an anonymized dataset, FFA Sora accurately simulates disease features from the input text, as confirmed by objective metrics: Frechet Video Distance (FVD) = 329.78, Learned Perceptual Image Patch Similarity (LPIPS) = 0.48, and Visual-question-answering Score (VQAScore) = 0.61. Specific evaluations showed acceptable alignment between the generated videos and textual prompts, with BERTScore of 0.35. Additionally, the model demonstrated strong privacy-preserving performance in retrieval evaluations, achieving an average Recall@K of 0.073. Human assessments indicated satisfactory visual quality, with an average score of 1.570(scale: 1 = best, 5 = worst). This model addresses privacy concerns associated with sharing large-scale FFA data and enhances medical education.