# (Mis)alignment Between Stance Expressed in Social Media Data and Public Opinion Surveys

arXiv: https://arxiv.org/abs/2109.01762 · HTML: https://mlsanity.com/paper/2109.01762/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2021.

## Built on by (most cited first, 8 of 8)

- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Beyond Digital "Echo Chambers": The Role of Viewpoint Diversity in Political Discussion](https://mlsanity.com/paper/2212.09056/) (2022)
- [Evolving linguistic divergence on polarizing social media](https://mlsanity.com/paper/2309.01659/) (2023)
- [Can Third-parties Read Our Emotions?](https://mlsanity.com/paper/2504.18673/) (2025)
- [Perspectives of Non-Expert Users on Cyber Security and Privacy: An Analysis of Online Discussions on Twitter](https://mlsanity.com/paper/2206.02156/) (2022)
- [Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study](https://mlsanity.com/paper/2404.03987/) (2024)
- [We Are in This Together: Quantifying Community Subjective Wellbeing and Resilience](https://mlsanity.com/paper/2208.10766/) (2022)
- [Dialectograms: Machine Learning Differences between Discursive Communities](https://mlsanity.com/paper/2302.05657/) (2023)

## Abstract (arXiv)

Stance detection, which aims to determine whether an individual is for or against a target concept, promises to uncover public opinion from large streams of social media data. Yet even human annotation of social media content does not always capture "stance" as measured by public opinion polls. We demonstrate this by directly comparing an individual's self-reported stance to the stance inferred from their social media data. Leveraging a longitudinal public opinion survey with respondent Twitter handles, we conducted this comparison for 1,129 individuals across four salient targets. We find that recall is high for both "Pro" and "Anti" stance classifications but precision is variable in a number of cases. We identify three factors leading to the disconnect between text and author stance: temporal inconsistencies, differences in constructs, and measurement errors from both survey respondents and annotators. By presenting a framework for assessing the limitations of stance detection models, this work provides important insight into what stance detection truly measures.