mlsanityData to 14 Oct 2026

LLM Analysis of 150+ years of German Parliamentary Debates on Migration Reveals Shift from Post-War Solidarity to Anti-Solidarity in the Last Decade

arXiv 2509.07274 ↗First posted 8 Sep 2025cs.CL cs.CY cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Migration has been a core topic in German political debate, from postwar expellee displacement to labor migration and recent refugee movements. Large-scale analysis of such political discourse has traditionally required extensive manual annotation, limiting coverage. Large language models (LLMs) offer a scalable alternative. Using a theory-driven annotation scheme, we examine how well LLMs annotate subtypes of solidarity and anti-solidarity in German parliamentary debates and whether the resulting labels support valid downstream inference. We first evaluate multiple LLMs across model size, prompting strategies, fine-tuning, historical versus contemporary data, and systematic errors. The strongest models, especially GPT-5 and gpt-oss-120B, achieve macro- F1 scores comparable to human agreement, although their systematic errors can bias downstream results. We therefore combine soft-label model outputs with Design-based Supervised Learning (DSL) to reduce bias in long-term trend estimates. Beyond the methodological evaluation, we interpret the resulting annotations from a social-scientific perspective across the Reichstag (1867- 1933), West German Bundestag (1949-1990) and Re-Unified German Bundestag (1990-2025) corpus to trace trends in solidarity and anti-solidarity toward migrants, with detailed analysis of postwar Germany (1949-1957) and contemporary Germany (2009-2025). We find relatively high levels of solidarity in the postwar period, especially in group-based and compassionate forms, and a marked rise in anti-solidarity since 2015. We argue that LLMs can support large-scale social-scientific text analysis, but their outputs require rigorous validation and, where systematic errors are present, statistical correction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 104 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

16 of 104 references.

References · 16

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2024Assessing Political Bias in Large Language Modelscited by 32
  3. 2024Political-LLM: Large Language Models in Political Sciencecited by 27
  4. 2024Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AIcited by 17
  5. 2025Unmasking Digital Falsehoods: A Comparative Analysis of LLM-Based Misinformation Detection Strategiescited by 16
  6. 2024Towards Evaluation Guidelines for Empirical Studies involving LLMscited by 15
  7. 2021Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biasescited by 13
  8. 2023Positioning Political Texts with Large Language Models by Asking and Averagingcited by 13
  9. 2023Towards Coding Social Science Datasets with Language Modelscited by 11
  10. 2024Can Large Language Models Detect Misinformation in Scientific News Reporting?cited by 9
  11. 2024Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for Political Textcited by 8
  12. 2024OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistantscited by 7
4 more
  1. 2022Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debatescited by 6
  2. 2019I Stand With You: Using Emojis to Study Solidarity in Crisis Eventscited by 5
  3. 2021Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Datasetcited by 4
  4. 2023PopBERT. Detecting populism and its host ideologies in the German Bundestagcited by 4

Built on by

Not cited yet.