# ConStance: Modeling Annotation Contexts to Improve Stance Classification

arXiv: https://arxiv.org/abs/1708.06309 · HTML: https://mlsanity.com/paper/1708.06309/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2017.

## Built on by (most cited first, 7 of 7)

- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [Stance Detection: A Practical Guide to Classifying Political Beliefs in Text](https://mlsanity.com/paper/2305.01723/) (2023)
- [Toward Annotator Group Bias in Crowdsourcing](https://mlsanity.com/paper/2110.08038/) (2021)
- [STEM: Unsupervised STructural EMbedding for Stance Detection](https://mlsanity.com/paper/2112.00712/) (2021)
- [PAWLS: PDF Annotation With Labels and Structure](https://mlsanity.com/paper/2101.10281/) (2021)
- [A Weakly Supervised Approach for Classifying Stance in Twitter Replies](https://mlsanity.com/paper/2103.07098/) (2021)
- [Implicit Design Choices and Their Impact on Emotion Recognition Model Development and Evaluation](https://mlsanity.com/paper/2309.03238/) (2023)

## Abstract (arXiv)

Manual annotations are a prerequisite for many applications of machine learning. However, weaknesses in the annotation process itself are easy to overlook. In particular, scholars often choose what information to give to annotators without examining these decisions empirically. For subjective tasks such as sentiment analysis, sarcasm, and stance detection, such choices can impact results. Here, for the task of political stance detection on Twitter, we show that providing too little context can result in noisy and uncertain annotations, whereas providing too strong a context may cause it to outweigh other signals. To characterize and reduce these biases, we develop ConStance, a general model for reasoning about annotations across information conditions. Given conflicting labels produced by multiple annotators seeing the same instances with different contexts, ConStance simultaneously estimates gold standard labels and also learns a classifier for new instances. We show that the classifier learned by ConStance outperforms a variety of baselines at predicting political stance, while the model's interpretable parameters shed light on the effects of each context.