# POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection

arXiv: https://arxiv.org/abs/2205.00619 · HTML: https://mlsanity.com/paper/2205.00619/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 114 references that are in the map.
- First posted on arXiv 2 May 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [ERNIE: Enhanced Representation through Knowledge Integration](https://mlsanity.com/paper/1904.09223/) (2019)
- [BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model](https://mlsanity.com/paper/1902.04094/) (2019)
- [FinBERT: A Pretrained Language Model for Financial Communications](https://mlsanity.com/paper/2006.08097/) (2020)
- [Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations](https://mlsanity.com/paper/2010.03640/) (2020)
- [Stance Detection with Bidirectional Conditional Encoding](https://mlsanity.com/paper/1606.05464/) (2016)
- [We Can Detect Your Bias: Predicting the Political Ideology of News Articles](https://mlsanity.com/paper/2010.05338/) (2020)
- [Mitigating Political Bias in Language Models Through Reinforced Calibration](https://mlsanity.com/paper/2104.14795/) (2021)
- [In Plain Sight: Media Bias Through the Lens of Factual Reporting](https://mlsanity.com/paper/1909.02670/) (2019)
- [Stance and Sentiment in Tweets](https://mlsanity.com/paper/1605.01655/) (2016)
- [Unsupervised User Stance Detection on Twitter](https://mlsanity.com/paper/1904.02000/) (2019)
- [MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection](https://mlsanity.com/paper/1606.03784/) (2016)
- [Inference Time Style Control for Summarization](https://mlsanity.com/paper/2104.01724/) (2021)
- [Stance Detection on Tweets: An SVM-based Approach](https://mlsanity.com/paper/1803.08910/) (2018)
- [Incorporating Count-Based Features into Pre-Trained Models for Improved Stance Detection](https://mlsanity.com/paper/2010.09078/) (2020)
- [Identifying hidden coalitions in the US House of Representatives by optimally partitioning signed networks based on generalized balance](https://mlsanity.com/paper/2105.01913/) (2021)
- [Deep Learning for Political Science](https://mlsanity.com/paper/2005.06540/) (2020)
- [Cross-Partisan Discussions on YouTube: Conservatives Talk to Liberals but Liberals Don't Talk to Conservatives](https://mlsanity.com/paper/2104.05365/) (2021)

## Built on by (most cited first, 30 of 51)

- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models](https://mlsanity.com/paper/2305.09955/) (2023)
- [MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions](https://mlsanity.com/paper/2403.07678/) (2024)
- [Introducing MBIB -- the first Media Bias Identification Benchmark Task and Dataset Collection](https://mlsanity.com/paper/2304.13148/) (2023)
- [The Power of LLM-Generated Synthetic Data for Stance Detection in Online Political Discussions](https://mlsanity.com/paper/2406.12480/) (2024)
- [IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators](https://mlsanity.com/paper/2402.00345/) (2024)
- [Modelling Political Coalition Negotiations Using LLM-based Agents](https://mlsanity.com/paper/2402.11712/) (2024)
- [The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias](https://mlsanity.com/paper/2312.16148/) (2023)
- [ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Self-Refinement](https://mlsanity.com/paper/2401.13218/) (2024)
- [Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articles](https://mlsanity.com/paper/2406.10773/) (2024)
- [MOKA: Moral Knowledge Augmentation for Moral Event Extraction](https://mlsanity.com/paper/2311.09733/) (2023)
- [When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries](https://mlsanity.com/paper/2410.09978/) (2024)
- [Generative Entity-to-Entity Stance Detection with Knowledge Graph Augmentation](https://mlsanity.com/paper/2211.01467/) (2022)
- [Predicting Sentence-Level Factuality of News and Bias of Media Outlets](https://mlsanity.com/paper/2301.11850/) (2023)
- [Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy](https://mlsanity.com/paper/2304.02247/) (2023)
- [Connecting the Dots in News Analysis: Bridging the Cross-Disciplinary Disparities in Media Bias and Framing](https://mlsanity.com/paper/2309.08069/) (2023)
- [Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance](https://mlsanity.com/paper/2403.00236/) (2024)
- [PAR: Political Actor Representation Learning with Social Context and Expert Knowledge](https://mlsanity.com/paper/2210.08362/) (2022)
- [Discourse Structures Guided Fine-grained Propaganda Identification](https://mlsanity.com/paper/2310.18544/) (2023)
- [PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encoder](https://mlsanity.com/paper/2505.24646/) (2025)
- [P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models](https://mlsanity.com/paper/2311.09741/) (2023)
- [Neutralizing the Narrative: AI-Powered Debiasing of Online News Articles](https://mlsanity.com/paper/2504.03520/) (2025)
- [Talking Point based Ideological Discourse Analysis in News Events](https://mlsanity.com/paper/2504.07400/) (2025)
- [Sentence-level Media Bias Analysis with Event Relation Graph](https://mlsanity.com/paper/2404.01722/) (2024)
- [A Survey on Moral Foundation Theory and Pre-Trained Language Models: Current Advances and Challenges](https://mlsanity.com/paper/2409.13521/) (2024)
- [All Things Considered: Detecting Partisan Events from News Media with Cross-Article Comparison](https://mlsanity.com/paper/2310.18827/) (2023)
- [Encoding Hierarchical Schema via Concept Flow for Multifaceted Ideology Detection](https://mlsanity.com/paper/2405.18974/) (2024)
- [Enabling Contextual Soft Moderation on Social Media through Contrastive Textual Deviation](https://mlsanity.com/paper/2407.20910/) (2024)
- [MGM: Global Understanding of Audience Overlap Graphs for Predicting the Factuality and the Bias of News Media](https://mlsanity.com/paper/2412.10467/) (2024)
- [Conflicts in Texts: Data, Implications and Challenges](https://mlsanity.com/paper/2504.19472/) (2025)

## Abstract (arXiv)

Ideology is at the core of political science research. Yet, there still does not exist general-purpose tools to characterize and predict ideology across different genres of text. To this end, we study Pretrained Language Models using novel ideology-driven pretraining objectives that rely on the comparison of articles on the same story written by media of different ideologies. We further collect a large-scale dataset, consisting of more than 3.6M political news articles, for pretraining. Our model POLITICS outperforms strong baselines and the previous state-of-the-art models on ideology prediction and stance detection tasks. Further analyses show that POLITICS is especially good at understanding long or formally written texts, and is also robust in few-shot learning scenarios.