# Using Word Embeddings to Analyze Protests News

arXiv: https://arxiv.org/abs/2203.05875 · HTML: https://mlsanity.com/paper/2203.05875/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 11 Mar 2022.

## Builds on (most cited first, 10 of 10)

- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task](https://mlsanity.com/paper/1711.01505/) (2017)
- [Expert-Augmented Machine Learning](https://mlsanity.com/paper/1903.09731/) (2019)
- [Overview of CLEF 2019 Lab ProtestNews: Extracting Protests from News in a Cross-context Setting](https://mlsanity.com/paper/2008.00345/) (2020)
- [Cross-context News Corpus for Protest Events related Knowledge Base Construction](https://mlsanity.com/paper/2008.00351/) (2020)
- [Multitask Models for Supervised Protests Detection in Texts](https://mlsanity.com/paper/2005.02954/) (2020)

## Abstract (arXiv)

The first two tasks of the CLEF 2019 ProtestNews events focused on distinguishing between protest and non-protest related news articles and sentences in a binary classification task. Among the submissions, two well performing models have been chosen in order to replace the existing word embeddings word2vec and FastTest with ELMo and DistilBERT. Unlike bag of words or earlier vector approaches, ELMo and DistilBERT represent words as a sequence of vectors by capturing the meaning based on contextual information in the text. Without changing the architecture of the original models other than the word embeddings, the implementation of DistilBERT improved the performance measured on the F1-Score of 0.66 compared to the FastText implementation. DistilBERT also outperformed ELMo in both tasks and models. Cleaning the datasets by removing stopwords and lemmatizing the words has been shown to make the models more generalizable across different contexts when training on a dataset with Indian news articles and evaluating the models on a dataset with news articles from China.