# Language Models Learn Metadata: Political Stance Detection Case Study

arXiv: https://arxiv.org/abs/2409.13756 · HTML: https://mlsanity.com/paper/2409.13756/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 15 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [From LLM to NMT: Advancing Low-Resource Machine Translation with Claude](https://mlsanity.com/paper/2404.13813/) (2024)
- [Topic-Specific Sentiment Analysis Can Help Identify Political Ideology](https://mlsanity.com/paper/1810.12897/) (2018)
- [Party Matters: Enhancing Legislative Embeddings with Author Attributes for Vote Prediction](https://mlsanity.com/paper/1805.08182/) (2018)
- [Dynamic Few-Shot Learning for Knowledge Graph Question Answering](https://mlsanity.com/paper/2407.01409/) (2024)
- [Zero-shot and Few-shot Generation Strategies for Artificial Clinical Records](https://mlsanity.com/paper/2403.08664/) (2024)
- [Multi-aspect Multilingual and Cross-lingual Parliamentary Speech Analysis](https://mlsanity.com/paper/2207.01054/) (2022)

## Abstract (arXiv)

Stance detection is a crucial NLP task with numerous applications in social science, from analyzing online discussions to assessing political campaigns. This paper investigates the optimal way to incorporate metadata into a political stance detection task. We demonstrate that previous methods combining metadata with language-based data for political stance detection have not fully utilized the metadata information; our simple baseline, using only party membership information, surpasses the current state-of-the-art. We then show that prepending metadata (e.g., party and policy) to political speeches performs best, outperforming all baselines, indicating that complex metadata inclusion systems may not learn the task optimally.