mlsanityData to 14 Oct 2026

Automated Annotation of Evolving Corpora for Augmenting Longitudinal Network Data: A Framework Integrating Large Language Models and Expert Knowledge

arXiv 2503.01672 ↗First posted 3 Mar 2025cs.CL cs.SIcited by 0

Not on the map. Why?

Abstract (arXiv)

Longitudinal network data are essential for analyzing political, economic, and social systems and processes. In political science, these datasets are often generated through human annotation or supervised machine learning applied to evolving corpora. However, as semantic contexts shift over time, inferring dynamic interaction types on emerging issues among a diverse set of entities poses significant challenges, particularly in maintaining timely and consistent annotations. This paper presents the Expert-Augmented LLM Annotation (EALA) approach, which leverages Large Language Models (LLMs) in combination with historically annotated data and expert-constructed codebooks to extrapolate and extend datasets into future periods. We evaluate the performance and reliability of EALA using a dataset of climate negotiations. Our findings demonstrate that EALA effectively predicts nuanced interactions between negotiation parties and captures the evolution of topics over time. At the same time, we identify several limitations inherent to LLM-based annotation, highlighting areas for further improvement. Given the wide availability of codebooks and annotated datasets, EALA holds substantial promise for advancing research in political science and beyond.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 43 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

18 of 43 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  8. 2023Can Large Language Models Transform Computational Social Science?cited by 250
  9. 2023Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speechcited by 166
  10. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 120
  11. 2023Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Codingcited by 82
  12. 2023A Benchmark for Learning to Translate a New Language from One Grammar Bookcited by 75
6 more
  1. 2023Large Language Models can Learn Rulescited by 69
  2. 2023Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Modelscited by 39
  3. 2024Teaching Large Language Models an Unseen Language on the Flycited by 30
  4. 2022FaiRR: Faithful and Robust Deductive Reasoning over Natural Languagecited by 29
  5. 2024Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labelscited by 25
  6. 2024Enhancing LLM-Based Text Classification in Political Science: Automatic Prompt Optimization and Dynamic Exemplar Selection for Few-Shot Learningcited by 5

Built on by

Not cited yet.