# Incorporating Dictionaries into a Neural Network Architecture to Extract COVID-19 Medical Concepts From Social Media

arXiv: https://arxiv.org/abs/2309.02188 · HTML: https://mlsanity.com/paper/2309.02188/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [A large-scale COVID-19 Twitter chatter dataset for open scientific research -- an international collaboration](https://mlsanity.com/paper/2004.03688/) (2020)
- [SwellShark: A Generative Model for Biomedical Named Entity Recognition without Labeled Data](https://mlsanity.com/paper/1704.06360/) (2017)
- [Self-Attention Gazetteer Embeddings for Named-Entity Recognition](https://mlsanity.com/paper/2004.04060/) (2020)
- [Multi-Task Pharmacovigilance Mining from Social Media Posts](https://mlsanity.com/paper/1801.06294/) (2018)
- [Improving Neural Named Entity Recognition with Gazetteers](https://mlsanity.com/paper/2003.03072/) (2020)
- [Neural Entity Recognition with Gazetteer based Fusion](https://mlsanity.com/paper/2105.13225/) (2021)
- [A Biomedically oriented automatically annotated Twitter COVID-19 Dataset](https://mlsanity.com/paper/2107.12565/) (2021)

## Abstract (arXiv)

We investigate the potential benefit of incorporating dictionary information into a neural network architecture for natural language processing. In particular, we make use of this architecture to extract several concepts related to COVID-19 from an on-line medical forum. We use a sample from the forum to manually curate one dictionary for each concept. In addition, we use MetaMap, which is a tool for extracting biomedical concepts, to identify a small number of semantic concepts. For a supervised concept extraction task on the forum data, our best model achieved a macro F1 score of 90%. A major difficulty in medical concept extraction is obtaining labelled data from which to build supervised models. We investigate the utility of our models to transfer to data derived from a different source in two ways. First for producing labels via weak learning and second to perform concept extraction. The dataset we use in this case comprises COVID-19 related tweets and we achieve an F1 score 81% for symptom concept extraction trained on weakly labelled data. The utility of our dictionaries is compared with a COVID-19 symptom dictionary that was constructed directly from Twitter. Further experiments that incorporate BERT and a COVID-19 version of BERTweet demonstrate that the dictionaries provide a commensurate result. Our results show that incorporating small domain dictionaries to deep learning models can improve concept extraction tasks. Moreover, models built using dictionaries generalize well and are transferable to different datasets on a similar task.