mlsanityData to 14 Oct 2026

Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance

arXiv 2403.00236 ↗First posted 1 Mar 2024cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We investigate the performance of LLM-based zero-shot stance detection on tweets. Using FlanT5-XXL, an instruction-tuned open-source LLM, with the SemEval 2016 Tasks 6A, 6B, and P-Stance datasets, we study the performance and its variations under different prompts and decoding strategies, as well as the potential biases of the model. We show that the zero-shot approach can match or outperform state-of-the-art benchmarks, including fine-tuned models. We provide various insights into its performance including the sensitivity to instructions and prompts, the decoding strategies, the perplexity of the prompts, and to negations and oppositions present in prompts. Finally, we ensure that the LLM has not been trained on test datasets, and identify a positivity bias which may partially explain the performance differences across decoding strategie

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 39 of its 72 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

39 of 72 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  10. 2018Datasheets for Datasetscited by 1,386
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
27 more
  1. 2017Dynamic Routing Between Capsulescited by 689
  2. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  3. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  4. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  5. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 263
  6. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  7. 2023ChatGPT: Jack of all trades, master of nonecited by 204
  8. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 150
  9. 2022TimeLMs: Diachronic Language Models from Twittercited by 131
  10. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 99
  11. 2020Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representationscited by 98
  12. 2020Stance Detection on Social Media: State of the Art and Trendscited by 95
  13. 2016Stance Detection with Bidirectional Conditional Encodingcited by 93
  14. 2022How would Stance Detection Techniques Evolve after the Launch of ChatGPT?cited by 76
  15. 2016Stance and Sentiment in Tweetscited by 62
  16. 2022POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detectioncited by 51
  17. 2020Detecting Stance in Media on Global Warmingcited by 48
  18. 2020X-Stance: A Multilingual Multi-Target Dataset for Stance Detectioncited by 43
  19. 2021Adversarial Learning for Zero-Shot Stance Detection on Social Mediacited by 37
  20. 2020Stance Detection Benchmark: How Robust Is Your Stance Detection?cited by 33
  21. 2020Stance Detection in Web and Social Media: A Comparative Studycited by 25
  22. 2014Human language reveals a universal positivity biascited by 23
  23. 2023Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classificationcited by 17
  24. 2020Multilingual Stance Detection: The Catalonia Independence Corpuscited by 16
  25. 2022Exploiting Sentiment and Common Sense for Zero-shot Stance Detectioncited by 16
  26. 2018The Age of Social Sensingcited by 2
  27. 2022Tweets2Stance: Users stance detection exploiting Zero-Shot Learning Algorithms on Tweetscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 2.

6 papers.

  1. 2024LLM Chain Ensembles for Scalable and Accurate Data Annotationcited by 11
  2. 2024Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of Offensivenesscited by 7
  3. 2024Label Set Optimization via Activation Distribution Kurtosis for Zero-shot Classification with Generative Modelscited by 0
  4. 2025What Helps Language Models Predict Human Beliefs: Demographics or Prior Stances?cited by 0
  5. 2026Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoningcited by 0
  6. 2026MoPrO-SD: Modular Prompt Optimization for Stance Detectioncited by 0