mlsanityData to 7 Oct 2026

The Quest for Early Detection of Retinal Disease: 3D CycleGAN-based Translation of Optical Coherence Tomography into Confocal Microscopy

arXiv 2408.04091 ↗First posted 7 Aug 2024eess.IV cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Optical coherence tomography (OCT) and confocal microscopy are pivotal in retinal imaging, offering distinct advantages and limitations. In vivo OCT offers rapid, non-invasive imaging but can suffer from clarity issues and motion artifacts, while ex vivo confocal microscopy, providing high-resolution, cellular-detailed color images, is invasive and raises ethical concerns. To bridge the benefits of both modalities, we propose a novel framework based on unsupervised 3D CycleGAN for translating unpaired in vivo OCT to ex vivo confocal microscopy images. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy. We also introduce a unique dataset, OCT2Confocal, comprising mouse OCT and confocal retinal images, facilitating the development of and establishing a benchmark for cross-modal image translation research. Our model has been evaluated both quantitatively and qualitatively, achieving Fr'echet Inception Distance (FID) scores of 0.766 and Kernel Inception Distance (KID) scores as low as 0.153, and leading subjective Mean Opinion Scores (MOS). Our model demonstrated superior image fidelity and quality with limited data over existing methods. Our approach effectively synthesizes color information from 3D confocal images, closely approximating target outcomes and suggesting enhanced potential for diagnostic and monitoring applications in ophthalmology.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 76 references that are in the map. First posted on arXiv 7 Aug 2024.

Builds on

27 of 76 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2018Demystifying MMD GANscited by 1,268
  11. 2016Tutorial on Variational Autoencoderscited by 404
  12. 2018MedGAN: Medical Image Translation using GANscited by 152
15 more
  1. 2016Unsupervised Cross-Domain Image Generationcited by 125
  2. 2018Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Networkcited by 97
  3. 2023Unpaired Image-to-Image Translation via Neural Schr"odinger Bridgecited by 96
  4. 2016Learning Dense Correspondence via 3D-guided Cycle Consistencycited by 91
  5. 2018Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Networkcited by 74
  6. 2017Towards Deeper Understanding of Variational Autoencoding Modelscited by 56
  7. 2018Generative Adversarial Networks for Image-to-Image Translation on Multi-Contrast MR Images - A Comparison of CycleGAN and UNITcited by 18
  8. 2020Multimodal Transfer Learning-based Approaches for Retinal Vascular Segmentationcited by 8
  9. 2022Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairscited by 8
  10. 2023Unpaired Image-to-Image Translation with Limited Data to Reveal Subtle Phenotypescited by 7
  11. 2018Adversarial Sparse-View CBCT Artifact Reductioncited by 5
  12. 2022Optimal Transport-based Graph Matching for 3D retinal OCT image registrationcited by 4
  13. 2022Objective quality assessment of medical images and videos: Review and challengescited by 3
  14. 2023OCT2Confocal: 3D CycleGAN based Translation of Retinal OCT Images to Confocal Microscopycited by 3
  15. 2020Generating Fundus Fluorescence Angiography Images from Structure Fundus Images Using Generative Adversarial Networkscited by 1

Built on by

Not cited yet.