# AE-GPT: Using Large Language Models to Extract Adverse Events from Surveillance Reports-A Use Case with Influenza Vaccine Adverse Events

arXiv: https://arxiv.org/abs/2309.16150 · HTML: https://mlsanity.com/paper/2309.16150/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 64 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1907.05774/) (2019)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [Patent Claim Generation by Fine-Tuning OpenAI GPT-2](https://mlsanity.com/paper/1907.02052/) (2019)
- [Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNet](https://mlsanity.com/paper/2102.08036/) (2021)
- [GPT3-to-plan: Extracting plans from text using GPT-3](https://mlsanity.com/paper/2106.07131/) (2021)
- [Multilevel Large Language Models for Everyone](https://mlsanity.com/paper/2307.13221/) (2023)
- [Knowledge-Infused Self Attention Transformers](https://mlsanity.com/paper/2306.13501/) (2023)

## Built on by (most cited first, 8 of 8)

- [Large Language Models Struggle in Token-Level Clinical Named Entity Recognition](https://mlsanity.com/paper/2407.00731/) (2024)
- [GAMedX: Generative AI-based Medical Entity Data Extractor Using Large Language Models](https://mlsanity.com/paper/2405.20585/) (2024)
- [Relation Extraction Using Large Language Models: A Case Study on Acupuncture Point Locations](https://mlsanity.com/paper/2404.05415/) (2024)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [Improving Entity Recognition Using Ensembles of Deep Learning and Fine-tuned Large Language Models: A Case Study on Adverse Event Extraction from Multiple Sources](https://mlsanity.com/paper/2406.18049/) (2024)
- [MultiADE: A Multi-domain Benchmark for Adverse Drug Event Extraction](https://mlsanity.com/paper/2405.18015/) (2024)
- [AI-VaxGuide: An Agentic RAG-Based LLM for Vaccination Decisions](https://mlsanity.com/paper/2507.03493/) (2025)
- [A Comparative Study of Recent Large Language Models on Generating Hospital Discharge Summaries for Lung Cancer Patients](https://mlsanity.com/paper/2411.03805/) (2024)

## Abstract (arXiv)

Though Vaccines are instrumental in global health, mitigating infectious diseases and pandemic outbreaks, they can occasionally lead to adverse events (AEs). Recently, Large Language Models (LLMs) have shown promise in effectively identifying and cataloging AEs within clinical reports. Utilizing data from the Vaccine Adverse Event Reporting System (VAERS) from 1990 to 2016, this study particularly focuses on AEs to evaluate LLMs' capability for AE extraction. A variety of prevalent LLMs, including GPT-2, GPT-3 variants, GPT-4, and Llama 2, were evaluated using Influenza vaccine as a use case. The fine-tuned GPT 3.5 model (AE-GPT) stood out with a 0.704 averaged micro F1 score for strict match and 0.816 for relaxed match. The encouraging performance of the AE-GPT underscores LLMs' potential in processing medical data, indicating a significant stride towards advanced AE detection, thus presumably generalizable to other AE extraction tasks.