mlsanityData to 7 Oct 2026

Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study

arXiv 2404.03987 ↗First posted 5 Apr 2024cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

For a viewpoint-diverse news recommender, identifying whether two news articles express the same viewpoint is essential. One way to determine "same or different" viewpoint is stance detection. In this paper, we investigate the robustness of operationalization choices for few-shot stance detection, with special attention to modelling stance across different topics. Our experiments test pre-registered hypotheses on stance detection. Specifically, we compare two stance task definitions (Pro/Con versus Same Side Stance), two LLM architectures (bi-encoding versus cross-encoding), and adding Natural Language Inference knowledge, with pre-trained RoBERTa models trained with shots of 100 examples from 7 different stance detection datasets. Some of our hypotheses and claims from earlier work can be confirmed, while others give more inconsistent results. The effect of the Same Side Stance definition on performance differs per dataset and is influenced by other modelling choices. We found no relationship between the number of training topics in the training shots and performance. In general, cross-encoding out-performs bi-encoding, and adding NLI training to our models gives considerable improvement, but these results are not consistent across all datasets. Our results indicate that it is essential to include multiple datasets and systematic modelling experiments when aiming to find robust modelling choices for the concept `stance'.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

35 of 59 references.

References · 35

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  6. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 254
  7. 2022Efficient Few-Shot Learning Without Promptscited by 135
  8. 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
  9. 2020Exploring and Predicting Transferability across NLP Taskscited by 126
  10. 2019Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claimscited by 73
  11. 2018Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networkscited by 58
  12. 2019Classification and Clustering of Arguments with Contextualized Word Embeddingscited by 57
23 more
  1. 2017The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrantscited by 55
  2. 2018Cross-Target Stance Classification with Self-Attention Networkscited by 40
  3. 2021Adversarial Learning for Zero-Shot Stance Detection on Social Mediacited by 36
  4. 2020Stance Detection Benchmark: How Robust Is Your Stance Detection?cited by 33
  5. 2021Cross-Domain Label-Adaptive Stance Detectioncited by 30
  6. 2021Preregistering NLP Researchcited by 24
  7. 2022Towards Climate Awareness in NLP Researchcited by 19
  8. 2022Cluster & Tune: Boost Cold Start Performance in Text Classificationcited by 13
  9. 2019STANCY: Stance Classification Based on Consistency Cuescited by 11
  10. 2022FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialoguecited by 11
  11. 2021Is Stance Detection Topic-Independent and Cross-topic Generalizable? -- A Reproduction Studycited by 9
  12. 2021(Mis)alignment Between Stance Expressed in Social Media Data and Public Opinion Surveyscited by 8
  13. 2022Towards Analyzing the Bias of News Recommender Systems Using Sentiment and Stance Detectioncited by 7
  14. 2022Will It Blend? Mixing Training Paradigms & Prompting for Argument Quality Predictioncited by 7
  15. 2021Operationalizing Framing to Support Multiperspective Recommendations of Opinion Piecescited by 6
  16. 2022Contextual information integration for stance detection via cross-attentioncited by 6
  17. 2023A Two-Sided Discussion of Preregistration of NLP Researchcited by 6
  18. 2024Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalizationcited by 6
  19. 2023Surveying (Dis)Parities and Concerns of Compute Hungry NLP Researchcited by 5
  20. 2023Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detectioncited by 4
  21. 2023Topic Ontologies for Argumentscited by 3
  22. 2020Same Side Stance Classification Task: Facilitating Argument Stance Classification by Fine-tuning a BERT Modelcited by 2
  23. 2021Tribrid: Stance Classification with Neural Inconsistency Detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Structuring the Space of Perspectivescited by 1