mlsanityData to 7 Oct 2026

Contextual information integration for stance detection via cross-attention

arXiv 2211.01874 ↗First posted 3 Nov 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Stance detection deals with identifying an author's stance towards a target. Most existing stance detection models are limited because they do not consider relevant contextual information which allows for inferring the stance correctly. Complementary context can be found in knowledge bases but integrating the context into pretrained language models is non-trivial due to the graph structure of standard knowledge bases. To overcome this, we explore an approach to integrate contextual information as text which allows for integrating contextual information from heterogeneous sources, such as structured knowledge sources and by prompting large language models. Our approach can outperform competitive baselines on a large and diverse stance detection benchmark in a cross-target setup, i.e. for targets unseen during training. We demonstrate that it is more robust to noisy context and can regularize for unwanted correlations between labels and target-specific vocabulary. Finally, it is independent of the pretrained language model in use.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 37 of its 82 references that are in the map. First posted on arXiv 3 Nov 2022.

Builds on

37 of 82 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  10. 2019How Can We Know What Language Models Know?cited by 953
  11. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  12. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
25 more
  1. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  2. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  3. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  4. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  5. 2020Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representationscited by 97
  6. 2016Stance Detection with Bidirectional Conditional Encodingcited by 93
  7. 2020Stance Detection on Social Media: State of the Art and Trendscited by 93
  8. 2018RumourEval 2019: Determining Rumour Veracity and Support for Rumourscited by 74
  9. 2019A Richly Annotated Corpus for Different Tasks in Automated Fact-Checkingcited by 74
  10. 2019Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claimscited by 73
  11. 2018Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networkscited by 58
  12. 2020Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformerscited by 57
  13. 2017The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrantscited by 55
  14. 2020Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twittercited by 53
  15. 2017Neural Natural Language Inference Models Enhanced with External Knowledgecited by 40
  16. 2018Cross-Target Stance Classification with Self-Attention Networkscited by 40
  17. 2021Adversarial Learning for Zero-Shot Stance Detection on Social Mediacited by 36
  18. 2020Stance Detection Benchmark: How Robust Is Your Stance Detection?cited by 33
  19. 2021Cross-Domain Label-Adaptive Stance Detectioncited by 30
  20. 2021Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Trainingcited by 28
  21. 2021Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemescited by 17
  22. 2021Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentationcited by 17
  23. 2019STANCY: Stance Classification Based on Consistency Cuescited by 11
  24. 2019Dissecting Content and Context in Argumentative Relation Analysiscited by 10
  25. 2021Is Stance Detection Topic-Independent and Cross-topic Generalizable? -- A Reproduction Studycited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 0; 2026: 1.

6 papers.

  1. 2024The Lou Dataset -- Exploring the Impact of Gender-Fair Language in German Text Classificationcited by 5
  2. 2024Putting Context in Context: the Impact of Discussion Structure on Text Classificationcited by 3
  3. 2024Cross-Target Stance Detection: A Survey of Techniques, Datasets, and Challengescited by 3
  4. 2024Enabling Contextual Soft Moderation on Social Media through Contrastive Textual Deviationcited by 2
  5. 2024Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Studycited by 1
  6. 2026Contextualized Prompting For Stance Detection On Social Mediacited by 0