# CephGPT-4: An Interactive Multimodal Cephalometric Measurement and Diagnostic System with Visual Large Language Model

arXiv: https://arxiv.org/abs/2307.07518 · HTML: https://mlsanity.com/paper/2307.07518/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 1 Jul 2023.

## Builds on (most cited first, 9 of 9)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [LLM-Pruner: On the Structural Pruning of Large Language Models](https://mlsanity.com/paper/2305.11627/) (2023)
- [PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering](https://mlsanity.com/paper/2305.10415/) (2023)
- [Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca](https://mlsanity.com/paper/2304.08177/) (2023)
- [HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge](https://mlsanity.com/paper/2304.06975/) (2023)
- [XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models](https://mlsanity.com/paper/2306.07971/) (2023)
- [OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue](https://mlsanity.com/paper/2306.12174/) (2023)

## Built on by (most cited first, 5 of 5)

- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [ControlLLM: Augment Language Models with Tools by Searching on Graphs](https://mlsanity.com/paper/2310.17796/) (2023)
- [Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review](https://mlsanity.com/paper/2311.01918/) (2023)
- [The Role of Language Models in Modern Healthcare: A Comprehensive Review](https://mlsanity.com/paper/2409.16860/) (2024)
- [Large AI Models in Dental Healthcare: From General-Purpose Systems to Domain-Specific Foundation Models](https://mlsanity.com/paper/2606.02914/) (2026)

## Abstract (arXiv)

Large-scale multimodal language models (LMMs) have achieved remarkable success in general domains. However, the exploration of diagnostic language models based on multimodal cephalometric medical data remains limited. In this paper, we propose a novel multimodal cephalometric analysis and diagnostic dialogue model. Firstly, a multimodal orthodontic medical dataset is constructed, comprising cephalometric images and doctor-patient dialogue data, with automatic analysis of cephalometric landmarks using U-net and generation of diagnostic reports. Then, the cephalometric dataset and generated diagnostic reports are separately fine-tuned on Minigpt-4 and VisualGLM. Results demonstrate that the CephGPT-4 model exhibits excellent performance and has the potential to revolutionize orthodontic measurement and diagnostic applications. These innovations hold revolutionary application potential in the field of orthodontics.