# Ophtha-LLaMA2: A Large Language Model for Ophthalmology

arXiv: https://arxiv.org/abs/2312.04906 · HTML: https://mlsanity.com/paper/2312.04906/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2305.17100/) (2023)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](https://mlsanity.com/paper/2303.11032/) (2023)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics](https://mlsanity.com/paper/2304.01938/) (2023)
- [AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiology](https://mlsanity.com/paper/2306.10095/) (2023)
- [Context Matters: A Strategy to Pre-train Language Model for Science Education](https://mlsanity.com/paper/2301.12031/) (2023)
- [Artificial General Intelligence for Medical Imaging Analysis](https://mlsanity.com/paper/2306.05480/) (2023)
- [ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT](https://mlsanity.com/paper/2304.11107/) (2023)
- [Putting ChatGPT's Medical Advice to the (Turing) Test](https://mlsanity.com/paper/2301.10035/) (2023)
- [MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering](https://mlsanity.com/paper/2309.16035/) (2023)
- [CohortGPT: An Enhanced GPT for Participant Recruitment in Clinical Study](https://mlsanity.com/paper/2307.11346/) (2023)
- [Matching Exemplar as Next Sentence Prediction (MeNSP): Zero-shot Prompt Learning for Automatic Scoring in Science Education](https://mlsanity.com/paper/2301.08771/) (2023)
- [ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data](https://mlsanity.com/paper/2310.05242/) (2023)
- [Exploring Multimodal Approaches for Alzheimer's Disease Detection Using Patient Speech Transcript and Audio Data](https://mlsanity.com/paper/2307.02514/) (2023)
- [Benchmarking a foundation LLM on its ability to re-label structure names in accordance with the AAPM TG-263 report](https://mlsanity.com/paper/2310.03874/) (2023)

## Built on by (most cited first, 9 of 9)

- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024)
- [Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research](https://mlsanity.com/paper/2412.04497/) (2024)
- [Large Language Models for Manufacturing](https://mlsanity.com/paper/2410.21418/) (2024)
- [Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis](https://mlsanity.com/paper/2402.11398/) (2024)
- [Multimodal Large Language Models for Medicine: A Comprehensive Survey](https://mlsanity.com/paper/2504.21051/) (2025)
- [Transcending Language Boundaries: Harnessing LLMs for Low-Resource Language Translation](https://mlsanity.com/paper/2411.11295/) (2024)
- [Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports](https://mlsanity.com/paper/2407.05758/) (2024)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [X-PCR: A Benchmark for Cross-modality Progressive Clinical Reasoning in Ophthalmic Diagnosis](https://mlsanity.com/paper/2604.20350/) (2026)

## Abstract (arXiv)

In recent years, pre-trained large language models (LLMs) have achieved tremendous success in the field of Natural Language Processing (NLP). Prior studies have primarily focused on general and generic domains, with relatively less research on specialized LLMs in the medical field. The specialization and high accuracy requirements for diagnosis in the medical field, as well as the challenges in collecting large-scale data, have constrained the application and development of LLMs in medical scenarios. In the field of ophthalmology, clinical diagnosis mainly relies on doctors' interpretation of reports and making diagnostic decisions. In order to take advantage of LLMs to provide decision support for doctors, we collected three modalities of ophthalmic report data and fine-tuned the LLaMA2 model, successfully constructing an LLM termed the "Ophtha-LLaMA2" specifically tailored for ophthalmic disease diagnosis. Inference test results show that even with a smaller fine-tuning dataset, Ophtha-LLaMA2 performs significantly better in ophthalmic diagnosis compared to other LLMs. It demonstrates that the Ophtha-LLaMA2 exhibits satisfying accuracy and efficiency in ophthalmic disease diagnosis, making it a valuable tool for ophthalmologists to provide improved diagnostic support for patients. This research provides a useful reference for the application of LLMs in the field of ophthalmology, while showcasing the immense potential and prospects in this domain.