# Is External Information Useful for Stance Detection with LLMs?

arXiv: https://arxiv.org/abs/2507.01543 · HTML: https://mlsanity.com/paper/2507.01543/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 22 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Builds on (most cited first, 6 of 6)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Stance Detection with Collaborative Role-Infused LLM-Based Agents](https://mlsanity.com/paper/2310.10467/) (2023)
- [Infusing Knowledge from Wikipedia to Enhance Stance Detection](https://mlsanity.com/paper/2204.03839/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Systematic Comparison of Prompting and Multi-Agent Methods for LLM-based Stance Detection](https://mlsanity.com/paper/2604.26319/) (2026)
- [Contextualized Prompting For Stance Detection On Social Media](https://mlsanity.com/paper/2606.06022/) (2026)
- [Not All or None: Dynamic Construction of Target-aware Memory Graph for Conversational Stance Detection](https://mlsanity.com/paper/2608.29066/) (2026)
- [MMDS-Bench: Benchmarking Multimodal Large Language Models on Dynamic Stance in Social Media Interactions](https://mlsanity.com/paper/2608.30903/) (2026)

## Abstract (arXiv)

In the stance detection task, a text is classified as either favorable, opposing, or neutral towards a target. Prior work suggests that the use of external information, e.g., excerpts from Wikipedia, improves stance detection performance. However, whether or not such information can benefit large language models (LLMs) remains an unanswered question, despite their wide adoption in many reasoning tasks. In this study, we conduct a systematic evaluation on how Wikipedia and web search external information can affect stance detection across eight LLMs and in three datasets with 12 targets. Surprisingly, we find that such information degrades performance in most cases, with macro F1 scores dropping by up to 27.9%. We explain this through experiments showing LLMs' tendency to align their predictions with the stance and sentiment of the provided information rather than the ground truth stance of the given text. We also find that performance degradation persists with chain-of-thought prompting, while fine-tuning mitigates but does not fully eliminate it. Our findings, in contrast to previous literature on BERT-based systems which suggests that external information enhances performance, highlight the risks of information biases in LLM-based stance classifiers. Code is available at https://github.com/ngqm/acl2025-stance-detection.