# Can ChatGPT be Your Personal Medical Assistant?

arXiv: https://arxiv.org/abs/2312.12006 · HTML: https://mlsanity.com/paper/2312.12006/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 5 of 5)

- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [ChatGPT: The End of Online Exam Integrity?](https://mlsanity.com/paper/2212.09292/) (2022)
- [Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models](https://mlsanity.com/paper/2303.13809/) (2023)
- [Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable information?](https://mlsanity.com/paper/2307.16382/) (2023)

## Built on by (most cited first, 2 of 2)

- [Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Images](https://mlsanity.com/paper/2405.20469/) (2024)
- [A Strategy to Combine 1stGen Transformers and Open LLMs for Automatic Text Classification](https://mlsanity.com/paper/2408.09629/) (2024)

## Abstract (arXiv)

The advanced large language model (LLM) ChatGPT has shown its potential in different domains and remains unbeaten due to its characteristics compared to other LLMs. This study aims to evaluate the potential of using a fine-tuned ChatGPT model as a personal medical assistant in the Arabic language. To do so, this study uses publicly available online questions and answering datasets in Arabic language. There are almost 430K questions and answers for 20 disease-specific categories. GPT-3.5-turbo model was fine-tuned with a portion of this dataset. The performance of this fine-tuned model was evaluated through automated and human evaluation. The automated evaluations include perplexity, coherence, similarity, and token count. Native Arabic speakers with medical knowledge evaluated the generated text by calculating relevance, accuracy, precision, logic, and originality. The overall result shows that ChatGPT has a bright future in medical assistance.