# Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study

arXiv: https://arxiv.org/abs/2402.15663 · HTML: https://mlsanity.com/paper/2402.15663/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 24 Feb 2024.

## Builds on (most cited first, 11 of 11)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [LLM-powered Data Augmentation for Enhanced Cross-lingual Performance](https://mlsanity.com/paper/2305.14288/) (2023)
- [PHEE: A Dataset for Pharmacovigilance Event Extraction from Text](https://mlsanity.com/paper/2210.12560/) (2022)
- [Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies](https://mlsanity.com/paper/2102.08366/) (2021)
- [NapSS: Paragraph-level Medical Text Simplification via Narrative Prompting and Sentence-matching Summarization](https://mlsanity.com/paper/2302.05574/) (2023)
- [Event Temporal Relation Extraction with Bayesian Translational Model](https://mlsanity.com/paper/2302.04985/) (2023)

## Built on by (most cited first, 12 of 12)

- [MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance](https://mlsanity.com/paper/2408.01869/) (2024)
- [REGen: A Reliable Evaluation Framework for Generative Event Argument Extraction](https://mlsanity.com/paper/2502.16838/) (2025)
- [Cascading Large Language Models for Salient Event Graph Generation](https://mlsanity.com/paper/2406.18449/) (2024)
- [SciGisPy: a Novel Metric for Biomedical Text Simplification via Gist Inference Score](https://mlsanity.com/paper/2410.09632/) (2024)
- [DrugWatch: A Comprehensive Multi-Source Data Visualisation Platform for Drug Safety Information](https://mlsanity.com/paper/2407.01585/) (2024)
- [Explaining Matters: Leveraging Definitions and Semantic Expansion for Sexism Detection](https://mlsanity.com/paper/2506.06238/) (2025)
- [MultiADE: A Multi-domain Benchmark for Adverse Drug Event Extraction](https://mlsanity.com/paper/2405.18015/) (2024)
- [SafeSpeech: A Comprehensive and Interactive Tool for Analysing Sexist and Abusive Language in Conversations](https://mlsanity.com/paper/2503.06534/) (2025)
- [DiVA-DocRE: A Discriminative and Voice-Aware Paradigm for Document-Level Relation Extraction](https://mlsanity.com/paper/2409.13717/) (2024)
- [Society of Medical Simplifiers](https://mlsanity.com/paper/2410.09631/) (2024)
- [RAG-based Architectures for Drug Side Effect Retrieval in LLMs](https://mlsanity.com/paper/2507.13822/) (2025)
- [Verb-ICL: Rethinking In-Context Learning for Structured Prediction](https://mlsanity.com/paper/2610.04725/) (2026)

## Abstract (arXiv)

With the advent of large language models (LLMs), there has been growing interest in exploring their potential for medical applications. This research aims to investigate the ability of LLMs, specifically ChatGPT, in the context of pharmacovigilance event extraction, of which the main goal is to identify and extract adverse events or potential therapeutic events from textual medical sources. We conduct extensive experiments to assess the performance of ChatGPT in the pharmacovigilance event extraction task, employing various prompts and demonstration selection strategies. The findings demonstrate that while ChatGPT demonstrates reasonable performance with appropriate demonstration selection strategies, it still falls short compared to fully fine-tuned small models. Additionally, we explore the potential of leveraging ChatGPT for data augmentation. However, our investigation reveals that the inclusion of synthesized data into fine-tuning may lead to a decrease in performance, possibly attributed to noise in the ChatGPT-generated labels. To mitigate this, we explore different filtering strategies and find that, with the proper approach, more stable performance can be achieved, although constant improvement remains elusive.