# Improving Adverse Drug Event Extraction with SpanBERT on Different Text Typologies

arXiv: https://arxiv.org/abs/2105.08882 · HTML: https://mlsanity.com/paper/2105.08882/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [An Effective Transition-based Model for Discontinuous NER](https://mlsanity.com/paper/2004.13454/) (2020)
- [Dissecting Span Identification Tasks with Performance Prediction](https://mlsanity.com/paper/2010.02587/) (2020)
- [Neural DrugNet](https://mlsanity.com/paper/1809.01500/) (2018)

## Built on by (most cited first, 7 of 7)

- [Exploring a Unified Sequence-To-Sequence Transformer for Medical Product Safety Monitoring in Social Media](https://mlsanity.com/paper/2109.05815/) (2021)
- [NADE: A Benchmark for Robust Adverse Drug Events Extraction in Face of Negations](https://mlsanity.com/paper/2109.10080/) (2021)
- [Increasing Adverse Drug Events extraction robustness on social media: case study on negation and speculation](https://mlsanity.com/paper/2209.02812/) (2022)
- [AILAB-Udine@SMM4H 22: Limits of Transformers and BERT Ensembles](https://mlsanity.com/paper/2209.03452/) (2022)
- [MultiADE: A Multi-domain Benchmark for Adverse Drug Event Extraction](https://mlsanity.com/paper/2405.18015/) (2024)
- [DistALANER: Distantly Supervised Active Learning Augmented Named Entity Recognition in the Open Source Software Ecosystem](https://mlsanity.com/paper/2402.16159/) (2024)
- [RA-MTR: A Retrieval Augmented Multi-Task Reader based Approach for Inspirational Quote Extraction from Long Documents](https://mlsanity.com/paper/2502.12124/) (2025)

## Abstract (arXiv)

In recent years, Internet users are reporting Adverse Drug Events (ADE) on social media, blogs and health forums. Because of the large volume of reports, pharmacovigilance is seeking to resort to NLP to monitor these outlets. We propose for the first time the use of the SpanBERT architecture for the task of ADE extraction: this new version of the popular BERT transformer showed improved capabilities with multi-token text spans. We validate our hypothesis with experiments on two datasets (SMM4H and CADEC) with different text typologies (tweets and blog posts), finding that SpanBERT combined with a CRF outperforms all the competitors on both of them.