# Stanceformer: Target-Aware Transformer for Stance Detection

arXiv: https://arxiv.org/abs/2410.07083 · HTML: https://mlsanity.com/paper/2410.07083/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 59 references that are in the map.
- First posted on arXiv 9 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations](https://mlsanity.com/paper/2010.03640/) (2020)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter](https://mlsanity.com/paper/2005.07503/) (2020)
- [Stance Detection with Bidirectional Conditional Encoding](https://mlsanity.com/paper/1606.05464/) (2016)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://mlsanity.com/paper/1704.05972/) (2017)
- [How would Stance Detection Techniques Evolve after the Launch of ChatGPT?](https://mlsanity.com/paper/2212.14548/) (2022)
- [RumourEval 2019: Determining Rumour Veracity and Support for Rumours](https://mlsanity.com/paper/1809.06683/) (2018)
- [Cross-Target Stance Classification with Self-Attention Networks](https://mlsanity.com/paper/1805.06593/) (2018)
- [Adversarial Learning for Zero-Shot Stance Detection on Social Media](https://mlsanity.com/paper/2105.06603/) (2021)
- [Infusing Knowledge from Wikipedia to Enhance Stance Detection](https://mlsanity.com/paper/2204.03839/) (2022)
- [Chain-of-Thought Embeddings for Stance Detection on Social Media](https://mlsanity.com/paper/2310.19750/) (2023)
- [Few-Shot Stance Detection via Target-Aware Prompt Distillation](https://mlsanity.com/paper/2206.13214/) (2022)
- [OpenStance: Real-world Zero-shot Stance Detection](https://mlsanity.com/paper/2210.14299/) (2022)

## Built on by (most cited first, 5 of 5)

- [Large Language Models Meet Stance Detection: A Survey of Tasks, Methods, Applications, Challenges and Future Directions](https://mlsanity.com/paper/2505.08464/) (2025)
- [StanceMoE: Mixture-of-Experts Architecture for Stance Detection](https://mlsanity.com/paper/2604.00878/) (2026)
- [Zero-Shot Stance Detection in the Wild: Dynamic Target Generation and Multi-Target Adaptation](https://mlsanity.com/paper/2601.19802/) (2026)
- [Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning](https://mlsanity.com/paper/2606.26571/) (2026)
- [A Social Media Analysis of Discourse on the Israel--Palestine Conflict on Telegram](https://mlsanity.com/paper/2608.21385/) (2026)

## Abstract (arXiv)

The task of Stance Detection involves discerning the stance expressed in a text towards a specific subject or target. Prior works have relied on existing transformer models that lack the capability to prioritize targets effectively. Consequently, these models yield similar performance regardless of whether we utilize or disregard target information, undermining the task's significance. To address this challenge, we introduce Stanceformer, a target-aware transformer model that incorporates enhanced attention towards the targets during both training and inference. Specifically, we design a textitTarget Awareness matrix that increases the self-attention scores assigned to the targets. We demonstrate the efficacy of the Stanceformer with various BERT-based models, including state-of-the-art models and Large Language Models (LLMs), and evaluate its performance across three stance detection datasets, alongside a zero-shot dataset. Our approach Stanceformer not only provides superior performance but also generalizes even to other domains, such as Aspect-based Sentiment Analysis. We make the code publicly available.footnotescriptsizeurlhttps://github.com/kgarg8/Stanceformer