# View Distillation with Unlabeled Data for Extracting Adverse Drug Effects from User-Generated Data

arXiv: https://arxiv.org/abs/2105.11354 · HTML: https://mlsanity.com/paper/2105.11354/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 24 May 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Did You Really Just Have a Heart Attack? Towards Robust Detection of Personal Health Mentions in Social Media](https://mlsanity.com/paper/1802.09130/) (2018)
- [Domain-Guided Task Decomposition with Self-Training for Detecting Personal Events in Social Media](https://mlsanity.com/paper/2004.10201/) (2020)
- [Mining Coronavirus (COVID-19) Posts in Social Media](https://mlsanity.com/paper/2004.06778/) (2020)

## Built on by (most cited first, 2 of 2)

- [Multi-View Active Learning for Short Text Classification in User-Generated Data](https://mlsanity.com/paper/2112.02611/) (2021)
- [Multiple-Source Domain Adaptation via Coordinated Domain Encoders and Paired Classifiers](https://mlsanity.com/paper/2201.11870/) (2022)

## Abstract (arXiv)

We present an algorithm based on multi-layer transformers for identifying Adverse Drug Reactions (ADR) in social media data. Our model relies on the properties of the problem and the characteristics of contextual word embeddings to extract two views from documents. Then a classifier is trained on each view to label a set of unlabeled documents to be used as an initializer for a new classifier in the other view. Finally, the initialized classifier in each view is further trained using the initial training examples. We evaluated our model in the largest publicly available ADR dataset. The experiments testify that our model significantly outperforms the transformer-based models pretrained on domain-specific data.