# Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study

arXiv: https://arxiv.org/abs/2404.03987 · HTML: https://mlsanity.com/paper/2404.03987/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 59 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Builds on (most cited first, 30 of 35)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [Efficient Few-Shot Learning Without Prompts](https://mlsanity.com/paper/2209.11055/) (2022)
- [Exploring and Predicting Transferability across NLP Tasks](https://mlsanity.com/paper/2005.00770/) (2020)
- [Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?](https://mlsanity.com/paper/2005.00628/) (2020)
- [Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims](https://mlsanity.com/paper/1906.03538/) (2019)
- [Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networks](https://mlsanity.com/paper/1802.05758/) (2018)
- [Classification and Clustering of Arguments with Contextualized Word Embeddings](https://mlsanity.com/paper/1906.09821/) (2019)
- [The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants](https://mlsanity.com/paper/1708.01425/) (2017)
- [Cross-Target Stance Classification with Self-Attention Networks](https://mlsanity.com/paper/1805.06593/) (2018)
- [Adversarial Learning for Zero-Shot Stance Detection on Social Media](https://mlsanity.com/paper/2105.06603/) (2021)
- [Stance Detection Benchmark: How Robust Is Your Stance Detection?](https://mlsanity.com/paper/2001.01565/) (2020)
- [Cross-Domain Label-Adaptive Stance Detection](https://mlsanity.com/paper/2104.07467/) (2021)
- [Preregistering NLP Research](https://mlsanity.com/paper/2103.06944/) (2021)
- [Towards Climate Awareness in NLP Research](https://mlsanity.com/paper/2205.05071/) (2022)
- [Cluster & Tune: Boost Cold Start Performance in Text Classification](https://mlsanity.com/paper/2203.10581/) (2022)
- [STANCY: Stance Classification Based on Consistency Cues](https://mlsanity.com/paper/1910.06048/) (2019)
- [FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue](https://mlsanity.com/paper/2205.06262/) (2022)
- [Is Stance Detection Topic-Independent and Cross-topic Generalizable? -- A Reproduction Study](https://mlsanity.com/paper/2110.07693/) (2021)
- [(Mis)alignment Between Stance Expressed in Social Media Data and Public Opinion Surveys](https://mlsanity.com/paper/2109.01762/) (2021)
- [Towards Analyzing the Bias of News Recommender Systems Using Sentiment and Stance Detection](https://mlsanity.com/paper/2203.05824/) (2022)
- [Will It Blend? Mixing Training Paradigms & Prompting for Argument Quality Prediction](https://mlsanity.com/paper/2209.08966/) (2022)
- [Operationalizing Framing to Support Multiperspective Recommendations of Opinion Pieces](https://mlsanity.com/paper/2101.06141/) (2021)
- [Contextual information integration for stance detection via cross-attention](https://mlsanity.com/paper/2211.01874/) (2022)
- [A Two-Sided Discussion of Preregistration of NLP Research](https://mlsanity.com/paper/2302.10086/) (2023)
- [Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalization](https://mlsanity.com/paper/2402.01375/) (2024)

## Built on by (most cited first, 1 of 1)

- [Structuring the Space of Perspectives](https://mlsanity.com/paper/2608.12113/) (2026)

## Abstract (arXiv)

For a viewpoint-diverse news recommender, identifying whether two news articles express the same viewpoint is essential. One way to determine "same or different" viewpoint is stance detection. In this paper, we investigate the robustness of operationalization choices for few-shot stance detection, with special attention to modelling stance across different topics. Our experiments test pre-registered hypotheses on stance detection. Specifically, we compare two stance task definitions (Pro/Con versus Same Side Stance), two LLM architectures (bi-encoding versus cross-encoding), and adding Natural Language Inference knowledge, with pre-trained RoBERTa models trained with shots of 100 examples from 7 different stance detection datasets. Some of our hypotheses and claims from earlier work can be confirmed, while others give more inconsistent results. The effect of the Same Side Stance definition on performance differs per dataset and is influenced by other modelling choices. We found no relationship between the number of training topics in the training shots and performance. In general, cross-encoding out-performs bi-encoding, and adding NLI training to our models gives considerable improvement, but these results are not consistent across all datasets. Our results indicate that it is essential to include multiple datasets and systematic modelling experiments when aiming to find robust modelling choices for the concept `stance'.