# Stance Detection Benchmark: How Robust Is Your Stance Detection?

arXiv: https://arxiv.org/abs/2001.01565 · HTML: https://mlsanity.com/paper/2001.01565/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2020.

## Built on by (most cited first, 30 of 33)

- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [Cross-Domain Label-Adaptive Stance Detection](https://mlsanity.com/paper/2104.07467/) (2021)
- [Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Training](https://mlsanity.com/paper/2109.06050/) (2021)
- [Improved Target-specific Stance Detection on Social Media Platforms by Delving into Conversation Threads](https://mlsanity.com/paper/2211.03061/) (2022)
- [Simulating Society Requires Simulating Thought](https://mlsanity.com/paper/2506.06958/) (2025)
- [MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection](https://mlsanity.com/paper/2109.08113/) (2021)
- [Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2201.00768/) (2022)
- [Multilingual Counter Narrative Type Classification](https://mlsanity.com/paper/2109.13664/) (2021)
- [Is Stance Detection Topic-Independent and Cross-topic Generalizable? -- A Reproduction Study](https://mlsanity.com/paper/2110.07693/) (2021)
- [AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking](https://mlsanity.com/paper/2104.13559/) (2021)
- [AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters](https://mlsanity.com/paper/2108.08103/) (2021)
- [Stanceosaurus: Classifying Stance Towards Multilingual Misinformation](https://mlsanity.com/paper/2210.15954/) (2022)
- [Contextual information integration for stance detection via cross-attention](https://mlsanity.com/paper/2211.01874/) (2022)
- [Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance](https://mlsanity.com/paper/2403.00236/) (2024)
- [Debiasing Stance Detection Models with Counterfactual Reasoning and Adversarial Bias Learning](https://mlsanity.com/paper/2212.10392/) (2022)
- [The COVMis-Stance dataset: Stance Detection on Twitter for COVID-19 Misinformation](https://mlsanity.com/paper/2204.02000/) (2022)
- [Diversity Over Size: On the Effect of Sample and Topic Sizes for Topic-Dependent Argument Mining Datasets](https://mlsanity.com/paper/2205.11472/) (2022)
- [Examining Temporalities on Stance Detection towards COVID-19 Vaccination](https://mlsanity.com/paper/2304.04806/) (2023)
- [Tribrid: Stance Classification with Neural Inconsistency Detection](https://mlsanity.com/paper/2109.06508/) (2021)
- [Language Independent Stance Detection: Social Interaction-based Embeddings and Large Language Models](https://mlsanity.com/paper/2210.05715/) (2022)
- [Robust Stance Detection: Understanding Public Perceptions in Social Media](https://mlsanity.com/paper/2309.15176/) (2023)
- [Stanceosaurus 2.0: Classifying Stance Towards Russian and Spanish Misinformation](https://mlsanity.com/paper/2402.03642/) (2024)
- [Enabling Contextual Soft Moderation on Social Media through Contrastive Textual Deviation](https://mlsanity.com/paper/2407.20910/) (2024)
- [Improving Stance Detection by Leveraging Measurement Knowledge from Social Sciences: A Case Study of Dutch Political Tweets and Traditional Gender Role Division](https://mlsanity.com/paper/2212.06543/) (2022)
- [Span Identification of Epistemic Stance-Taking in Academic Written English](https://mlsanity.com/paper/2306.02038/) (2023)
- [Multi-Task Learning Improves Performance In Deep Argument Mining Models](https://mlsanity.com/paper/2307.01401/) (2023)
- [Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study](https://mlsanity.com/paper/2404.03987/) (2024)
- [Zero-shot Cross-lingual Stance Detection via Adversarial Language Adaptation](https://mlsanity.com/paper/2404.14339/) (2024)
- [Catch Me If You Can: Deceiving Stance Detection and Geotagging Models to Protect Privacy of Individuals on Twitter](https://mlsanity.com/paper/2207.11500/) (2022)
- [Perturbations and Subpopulations for Testing Robustness in Token-Based Argument Unit Recognition](https://mlsanity.com/paper/2209.14780/) (2022)

## Abstract (arXiv)

Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim and has become a key component in applications like fake news detection, claim validation, and argument search. However, while stance is easily detected by humans, machine learning models are clearly falling short of this task. Given the major differences in dataset sizes and framing of StD (e.g. number of classes and inputs), we introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning (MDL) setting, as well as from related tasks via transfer learning. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets. Yet, the models still perform well below human capabilities and even simple adversarial attacks severely hurt the performance of MDL models. Deeper investigation into this phenomenon suggests the existence of biases inherited from multiple datasets by design. Our analysis emphasizes the need of focus on robustness and de-biasing strategies in multi-task learning approaches. The benchmark dataset and code is made available.