Evaluating ChatGPT text-mining of clinical records for obesity monitoring
Not on the map. Why?
Abstract (arXiv)
Background: Veterinary clinical narratives remain a largely untapped resource for addressing complex diseases. Here we compare the ability of a large language model (ChatGPT) and a previously developed regular expression (RegexT) to identify overweight body condition scores (BCS) in veterinary narratives. Methods: BCS values were extracted from 4,415 anonymised clinical narratives using either RegexT or by appending the narrative to a prompt sent to ChatGPT coercing the model to return the BCS information. Data were manually reviewed for comparison. Results: The precision of RegexT was higher (100%, 95% CI 94.81-100%) than the ChatGPT (89.3%; 95% CI82.75-93.64%). However, the recall of ChatGPT (100%. 95% CI 96.18-100%) was considerably higher than that of RegexT (72.6%, 95% CI 63.92-79.94%). Limitations: Subtle prompt engineering is needed to improve ChatGPT output. Conclusions: Large language models create diverse opportunities and, whilst complex, present an intuitive interface to information but require careful implementation to avoid unpredictable errors.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 15 references that are in the map. First posted on arXiv 3 Aug 2023.
Builds on
6 of 15 references.
References · 6
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
- 2020Recipes for building an open-domain chatbotcited by 593
- 2023GPT-NER: Named Entity Recognition via Large Language Modelscited by 160
- 2019TENER: Adapting Transformer Encoder for Named Entity Recognitioncited by 69
Built on by
Citing papers per year: 2024: 1.
1 paper.