# Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Training

arXiv: https://arxiv.org/abs/2109.06050 · HTML: https://mlsanity.com/paper/2109.06050/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 81 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Builds on (most cited first, 30 of 42)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [How Many Data Points is a Prompt Worth?](https://mlsanity.com/paper/2103.08493/) (2021)
- [Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models](https://mlsanity.com/paper/2106.13353/) (2021)
- [Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification](https://mlsanity.com/paper/2010.13641/) (2020)
- [Improving and Simplifying Pattern Exploiting Training](https://mlsanity.com/paper/2103.11955/) (2021)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://mlsanity.com/paper/1704.05972/) (2017)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [What to Pre-Train on? Efficient Intermediate Task Selection](https://mlsanity.com/paper/2104.08247/) (2021)
- [Unsupervised User Stance Detection on Twitter](https://mlsanity.com/paper/1904.02000/) (2019)
- [X-Stance: A Multilingual Multi-Target Dataset for Stance Detection](https://mlsanity.com/paper/2003.08385/) (2020)
- [Integrating Stance Detection and Fact Checking in a Unified Corpus](https://mlsanity.com/paper/1804.08012/) (2018)

## Built on by (most cited first, 28 of 28)

- [Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting](https://mlsanity.com/paper/2203.07856/) (2022)
- [The Emergence of Abstract Thought in Large Language Models Beyond Any Language](https://mlsanity.com/paper/2506.09890/) (2025)
- [Exploiting Sentiment and Common Sense for Zero-shot Stance Detection](https://mlsanity.com/paper/2208.08797/) (2022)
- [Improved Target-specific Stance Detection on Social Media Platforms by Delving into Conversation Threads](https://mlsanity.com/paper/2211.03061/) (2022)
- [Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models](https://mlsanity.com/paper/2205.02023/) (2022)
- [Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection](https://mlsanity.com/paper/2309.16424/) (2023)
- [Few-Shot Stance Detection via Target-Aware Prompt Distillation](https://mlsanity.com/paper/2206.13214/) (2022)
- [Stanceosaurus: Classifying Stance Towards Multilingual Misinformation](https://mlsanity.com/paper/2210.15954/) (2022)
- [Contextual information integration for stance detection via cross-attention](https://mlsanity.com/paper/2211.01874/) (2022)
- [Large Language Models Meet Stance Detection: A Survey of Tasks, Methods, Applications, Challenges and Future Directions](https://mlsanity.com/paper/2505.08464/) (2025)
- [Chain of Stance: Stance Detection with Large Language Models](https://mlsanity.com/paper/2408.04649/) (2024)
- [The COVMis-Stance dataset: Stance Detection on Twitter for COVID-19 Misinformation](https://mlsanity.com/paper/2204.02000/) (2022)
- [Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection](https://mlsanity.com/paper/2306.00765/) (2023)
- [Language Independent Stance Detection: Social Interaction-based Embeddings and Large Language Models](https://mlsanity.com/paper/2210.05715/) (2022)
- [Why Should This Article Be Deleted? Transparent Stance Detection in Multilingual Wikipedia Editor Discussions](https://mlsanity.com/paper/2310.05779/) (2023)
- [Effective Transfer Learning for Low-Resource Natural Language Understanding](https://mlsanity.com/paper/2208.09180/) (2022)
- [Explaining Search Result Stances to Opinionated People](https://mlsanity.com/paper/2309.08460/) (2023)
- [Zero-shot Cross-lingual Stance Detection via Adversarial Language Adaptation](https://mlsanity.com/paper/2404.14339/) (2024)
- [Tracking the Takes and Trajectories of English-Language News Narratives across Trustworthy and Worrisome Websites](https://mlsanity.com/paper/2501.09102/) (2025)
- [Intermediate-Task Transfer Learning: Leveraging Sarcasm Detection for Stance Detection](https://mlsanity.com/paper/2503.03172/) (2025)
- [Sarcasm Detection as a Catalyst: Improving Stance Detection with Cross-Target Capabilities](https://mlsanity.com/paper/2503.03787/) (2025)
- [Emotion-aware Dual Cross-Attentive Neural Network with Label Fusion for Stance Detection in Misinformative Social Media Content](https://mlsanity.com/paper/2505.23812/) (2025)
- [Zero-Shot Stance Detection in the Wild: Dynamic Target Generation and Multi-Target Adaptation](https://mlsanity.com/paper/2601.19802/) (2026)
- [Stance Detection and Open Research Avenues](https://mlsanity.com/paper/2210.12383/) (2022)
- [SCRum-9: Multilingual Stance Classification over Rumours on Social Media](https://mlsanity.com/paper/2505.18916/) (2025)
- [Multilingual Target-Stance Extraction](https://mlsanity.com/paper/2510.22334/) (2025)
- [Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning](https://mlsanity.com/paper/2606.26571/) (2026)
- [Rationale-Guided Knowledge Distillation for Cross-Lingual Stance Detection](https://mlsanity.com/paper/2607.18693/) (2026)

## Abstract (arXiv)

The goal of stance detection is to determine the viewpoint expressed in a piece of text towards a target. These viewpoints or contexts are often expressed in many different languages depending on the user and the platform, which can be a local news outlet, a social media platform, a news forum, etc. Most research in stance detection, however, has been limited to working with a single language and on a few limited targets, with little work on cross-lingual stance detection. Moreover, non-English sources of labelled data are often scarce and present additional challenges. Recently, large multilingual language models have substantially improved the performance on many non-English tasks, especially such with limited numbers of examples. This highlights the importance of model pre-training and its ability to learn from few examples. In this paper, we present the most comprehensive study of cross-lingual stance detection to date: we experiment with 15 diverse datasets in 12 languages from 6 language families, and with 6 low-resource evaluation settings each. For our experiments, we build on pattern-exploiting training, proposing the addition of a novel label encoder to simplify the verbalisation procedure. We further propose sentiment-based generation of stance data for pre-training, which shows sizeable improvement of more than 6% F1 absolute in low-shot settings compared to several strong baselines.