# Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization

arXiv: https://arxiv.org/abs/1907.12023 · HTML: https://mlsanity.com/paper/1907.12023/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2019.

## Built on by (most cited first, 11 of 11)

- [Multi-Modal Multi-Instance Learning for Retinal Disease Recognition](https://mlsanity.com/paper/2109.12307/) (2021)
- [MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition from Fundus Images](https://mlsanity.com/paper/2412.09402/) (2024)
- [Fundus-Enhanced Disease-Aware Distillation Model for Retinal Disease Classification from OCT Images](https://mlsanity.com/paper/2308.00291/) (2023)
- [Confidence-aware multi-modality learning for eye disease screening](https://mlsanity.com/paper/2405.18167/) (2024)
- [Unsupervised Domain Expansion for Visual Categorization](https://mlsanity.com/paper/2104.00233/) (2021)
- [A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models](https://mlsanity.com/paper/2508.03734/) (2025)
- [Masked Image Modelling for retinal OCT understanding](https://mlsanity.com/paper/2405.14788/) (2024)
- [KFWC: A Knowledge-Driven Deep Learning Model for Fine-grained Classification of Wet-AMD](https://mlsanity.com/paper/2112.12386/) (2021)
- [Cross-Fundus Transformer for Multi-modal Diabetic Retinopathy Grading with Cataract](https://mlsanity.com/paper/2411.00726/) (2024)
- [Lesion Localization in OCT by Semi-Supervised Object Detection](https://mlsanity.com/paper/2204.11227/) (2022)
- [Multi-modal and Multi-view Fundus Image Fusion for Retinopathy Diagnosis via Multi-scale Cross-attention and Shifted Window Self-attention](https://mlsanity.com/paper/2504.09106/) (2025)

## Abstract (arXiv)

This paper studies automated categorization of age-related macular degeneration (AMD) given a multi-modal input, which consists of a color fundus image and an optical coherence tomography (OCT) image from a specific eye. Previous work uses a traditional method, comprised of feature extraction and classifier training that cannot be optimized jointly. By contrast, we propose a two-stream convolutional neural network (CNN) that is end-to-end. The CNN's fusion layer is tailored to the need of fusing information from the fundus and OCT streams. For generating more multi-modal training instances, we introduce Loose Pair training, where a fundus image and an OCT image are paired based on class labels rather than eyes. Moreover, for a visual interpretation of how the individual modalities make contributions, we extend the class activation mapping technique to the multi-modal scenario. Experiments on a real-world dataset collected from an outpatient clinic justify the viability of our proposal for multi-modal AMD categorization.