# Few-Shot Stance Detection via Target-Aware Prompt Distillation

arXiv: https://arxiv.org/abs/2206.13214 · HTML: https://mlsanity.com/paper/2206.13214/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [How Many Data Points is a Prompt Worth?](https://mlsanity.com/paper/2103.08493/) (2021)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networks](https://mlsanity.com/paper/1802.05758/) (2018)
- [Cross-Domain Label-Adaptive Stance Detection](https://mlsanity.com/paper/2104.07467/) (2021)
- [Universal Text Representation from BERT: An Empirical Study](https://mlsanity.com/paper/1910.07973/) (2019)
- [Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Training](https://mlsanity.com/paper/2109.06050/) (2021)
- [Open Aspect Target Sentiment Classification with Natural Language Prompts](https://mlsanity.com/paper/2109.03685/) (2021)
- [STANCY: Stance Classification Based on Consistency Cues](https://mlsanity.com/paper/1910.06048/) (2019)
- [Topical Stance Detection for Twitter: A Two-Phase LSTM Model Using Attention](https://mlsanity.com/paper/1801.03032/) (2018)
- [DAN: Dual-View Representation Learning for Adapting Stance Classifiers to New Domains](https://mlsanity.com/paper/2003.06514/) (2020)

## Built on by (most cited first, 9 of 9)

- [Large Language Models Meet Stance Detection: A Survey of Tasks, Methods, Applications, Challenges and Future Directions](https://mlsanity.com/paper/2505.08464/) (2025)
- [Stanceformer: Target-Aware Transformer for Stance Detection](https://mlsanity.com/paper/2410.07083/) (2024)
- [Cross-Target Stance Detection: A Survey of Techniques, Datasets, and Challenges](https://mlsanity.com/paper/2409.13594/) (2024)
- [Relative Counterfactual Contrastive Learning for Mitigating Pretrained Stance Bias in Stance Detection](https://mlsanity.com/paper/2405.10991/) (2024)
- [Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks](https://mlsanity.com/paper/2312.08726/) (2023)
- [SocialPET: Socially Informed Pattern Exploiting Training for Few-Shot Stance Detection in Social Media](https://mlsanity.com/paper/2403.05216/) (2024)
- [CoSD: Collaborative Stance Detection with Contrastive Heterogeneous Topic Graph Learning](https://mlsanity.com/paper/2404.17609/) (2024)
- [MLSD: A Novel Few-Shot Learning Approach to Enhance Cross-Target and Cross-Domain Stance Detection](https://mlsanity.com/paper/2509.03725/) (2025)
- [Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning](https://mlsanity.com/paper/2606.26571/) (2026)

## Abstract (arXiv)

Stance detection aims to identify whether the author of a text is in favor of, against, or neutral to a given target. The main challenge of this task comes two-fold: few-shot learning resulting from the varying targets and the lack of contextual information of the targets. Existing works mainly focus on solving the second issue by designing attention-based models or introducing noisy external knowledge, while the first issue remains under-explored. In this paper, inspired by the potential capability of pre-trained language models (PLMs) serving as knowledge bases and few-shot learners, we propose to introduce prompt-based fine-tuning for stance detection. PLMs can provide essential contextual information for the targets and enable few-shot learning via prompts. Considering the crucial role of the target in stance detection task, we design target-aware prompts and propose a novel verbalizer. Instead of mapping each label to a concrete word, our verbalizer maps each label to a vector and picks the label that best captures the correlation between the stance and the target. Moreover, to alleviate the possible defect of dealing with varying targets with a single hand-crafted prompt, we propose to distill the information learned from multiple prompts. Experimental results show the superior performance of our proposed model in both full-data and few-shot scenarios.