# Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else?

arXiv: https://arxiv.org/abs/2111.05139 · HTML: https://mlsanity.com/paper/2111.05139/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 46 of its 104 references that are in the map.
- First posted on arXiv 9 Nov 2021.

## Builds on (most cited first, 30 of 46)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Stance Detection with Bidirectional Conditional Encoding](https://mlsanity.com/paper/1606.05464/) (2016)
- [Fighting an Infodemic: COVID-19 Fake News Dataset](https://mlsanity.com/paper/2011.03327/) (2020)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims](https://mlsanity.com/paper/1906.03538/) (2019)
- [A Survey on Natural Language Processing for Fake News Detection](https://mlsanity.com/paper/1811.00770/) (2018)
- [Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification](https://mlsanity.com/paper/1908.11860/) (2019)
- [Stance and Sentiment in Tweets](https://mlsanity.com/paper/1605.01655/) (2016)
- [A Joint Training Dual-MRC Framework for Aspect Based Sentiment Analysis](https://mlsanity.com/paper/2101.00816/) (2021)
- [Exploiting BERT for End-to-End Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1910.00883/) (2019)
- [The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants](https://mlsanity.com/paper/1708.01425/) (2017)
- [Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification](https://mlsanity.com/paper/1906.03820/) (2019)
- [A Retrospective Analysis of the Fake News Challenge Stance Detection Task](https://mlsanity.com/paper/1806.05180/) (2018)
- [Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media](https://mlsanity.com/paper/2007.07997/) (2020)
- [Cross-Target Stance Classification with Self-Attention Networks](https://mlsanity.com/paper/1805.06593/) (2018)
- [The Role of User Profile for Fake News Detection](https://mlsanity.com/paper/1904.13355/) (2019)
- [Combating Fake News: A Survey on Identification and Mitigation Techniques](https://mlsanity.com/paper/1901.06437/) (2019)
- [SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoods](https://mlsanity.com/paper/1610.03771/) (2016)
- [Hierarchical Propagation Networks for Fake News Detection: Investigation and Exploitation](https://mlsanity.com/paper/1903.09196/) (2019)
- [Characterizing COVID-19 Misinformation Communities Using a Novel Twitter Dataset](https://mlsanity.com/paper/2008.00791/) (2020)
- [Fake News Early Detection: An Interdisciplinary Study](https://mlsanity.com/paper/1904.11679/) (2019)

## Abstract (arXiv)

Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, however, the canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system. This paper aims to determine which of these techniques is best suited for this purpose and how each technique might best be used towards this end. Training datasets of the same size and nearly identical neural architectures (a BERT transformer as a word embedder with a single feed-forward layer thereafter) are used for each approach, which are then tested on sentiment- and stance-specific datasets to establish a baseline of how well each method can be used to do the other tasks. Four different datasets relating to COVID-19 disinformation are used to test the ability of each technique to detect disinformation on a topic that did not appear in the training data set. Quantitative and qualitative results from these tests are then used to provide insight into how best to employ these techniques in practice.