# MultiADE: A Multi-domain Benchmark for Adverse Drug Event Extraction

arXiv: https://arxiv.org/abs/2405.18015 · HTML: https://mlsanity.com/paper/2405.18015/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 73 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [Empirical Study of Zero-Shot NER with ChatGPT](https://mlsanity.com/paper/2310.10035/) (2023)
- [PHEE: A Dataset for Pharmacovigilance Event Extraction from Text](https://mlsanity.com/paper/2210.12560/) (2022)
- [An Effective Transition-based Model for Discontinuous NER](https://mlsanity.com/paper/2004.13454/) (2020)
- [Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study](https://mlsanity.com/paper/2402.15663/) (2024)
- [Exploring a Unified Sequence-To-Sequence Transformer for Medical Product Safety Monitoring in Social Media](https://mlsanity.com/paper/2109.05815/) (2021)
- [On-the-fly Definition Augmentation of LLMs for Biomedical NER](https://mlsanity.com/paper/2404.00152/) (2024)
- [AE-GPT: Using Large Language Models to Extract Adverse Events from Surveillance Reports-A Use Case with Influenza Vaccine Adverse Events](https://mlsanity.com/paper/2309.16150/) (2023)
- [Improving Adverse Drug Event Extraction with SpanBERT on Different Text Typologies](https://mlsanity.com/paper/2105.08882/) (2021)
- [Extensive Evaluation of Transformer-based Architectures for Adverse Drug Events Extraction](https://mlsanity.com/paper/2306.05276/) (2023)
- [Detecting Entities in the Astrophysics Literature: A Comparison of Word-based and Span-based Entity Recognition Methods](https://mlsanity.com/paper/2211.13819/) (2022)
- [Universal Domain Adaptation for Robust Handling of Distributional Shifts in NLP](https://mlsanity.com/paper/2310.14849/) (2023)
- [Cross-lingual Approaches for the Detection of Adverse Drug Reactions in German from a Patient's Perspective](https://mlsanity.com/paper/2208.02031/) (2022)

## Built on by (most cited first, 1 of 1)

- [Adverse Event Extraction from Discharge Summaries: A New Dataset, Annotation Scheme, and Initial Findings](https://mlsanity.com/paper/2506.14900/) (2025)

## Abstract (arXiv)

Active adverse event surveillance monitors Adverse Drug Events (ADE) from different data sources, such as electronic health records, medical literature, social media and search engine logs. Over the years, many datasets have been created, and shared tasks have been organised to facilitate active adverse event surveillance. However, most - if not all - datasets or shared tasks focus on extracting ADEs from a particular type of text. Domain generalisation - the ability of a machine learning model to perform well on new, unseen domains (text types) - is under-explored. Given the rapid advancements in natural language processing, one unanswered question is how far we are from having a single ADE extraction model that is effective on various types of text, such as scientific literature and social media posts. We contribute to answering this question by building a multi-domain benchmark for adverse drug event extraction, which we named MultiADE. The new benchmark comprises several existing datasets sampled from different text types and our newly created dataset - CADECv2, which is an extension of CADEC, covering online posts regarding more diverse drugs than CADEC. Our new dataset is carefully annotated by human annotators following detailed annotation guidelines. Our benchmark results show that the generalisation of the trained models is far from perfect, making it infeasible to be deployed to process different types of text. In addition, although intermediate transfer learning is a promising approach to utilising existing resources, further investigation is needed on methods of domain adaptation, particularly cost-effective methods to select useful training instances. The newly created CADECv2 and the scripts for building the benchmark are publicly available at CSIRO's Data Portal.