# Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification

arXiv: https://arxiv.org/abs/2308.07876 · HTML: https://mlsanity.com/paper/2308.07876/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 75 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Document-Level Event Argument Extraction by Conditional Generation](https://mlsanity.com/paper/2104.05919/) (2021)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction](https://mlsanity.com/paper/2109.03659/) (2021)
- [Zero-Shot Transfer Learning for Event Extraction](https://mlsanity.com/paper/1707.01066/) (2017)
- [Can we trust the evaluation on ChatGPT?](https://mlsanity.com/paper/2303.12767/) (2023)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [OntoZSL: Ontology-enhanced Zero-shot Learning](https://mlsanity.com/paper/2102.07339/) (2021)
- [Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encoding](https://mlsanity.com/paper/2005.06579/) (2020)
- [Neural models of factuality](https://mlsanity.com/paper/1804.02472/) (2018)
- [The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing](https://mlsanity.com/paper/2106.08037/) (2021)
- [Language Model Priming for Cross-Lingual Event Extraction](https://mlsanity.com/paper/2109.12383/) (2021)
- [Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2021): Workshop and Shared Task Report](https://mlsanity.com/paper/2108.07865/) (2021)
- [Few-Shot Upsampling for Protest Size Detection](https://mlsanity.com/paper/2105.11260/) (2021)
- [Controllable Fake Document Infilling for Cyber Deception](https://mlsanity.com/paper/2210.09917/) (2022)
- [A Monte Carlo Language Model Pipeline for Zero-Shot Sociopolitical Event Extraction](https://mlsanity.com/paper/2305.15051/) (2023)

## Built on by (most cited first, 2 of 2)

- [When Better Codebooks Are Not Enough: Predictive Performance and Behavioral Reliability in LLM Political Event Coding](https://mlsanity.com/paper/2606.06781/) (2026)
- [The Influence of Text Variation on User Engagement in Cross-Platform Content Sharing](https://mlsanity.com/paper/2505.03769/) (2025)

## Abstract (arXiv)

Is it possible accurately classify political relations within evolving event ontologies without extensive annotations? This study investigates zero-shot learning methods that use expert knowledge from existing annotation codebook, and evaluates the performance of advanced ChatGPT (GPT-3.5/4) and a natural language inference (NLI)-based model called ZSP. ChatGPT uses codebook's labeled summaries as prompts, whereas ZSP breaks down the classification task into context, event mode, and class disambiguation to refine task-specific hypotheses. This decomposition enhances interpretability, efficiency, and adaptability to schema changes. The experiments reveal ChatGPT's strengths and limitations, and crucially show ZSP's outperformance of dictionary-based methods and its competitive edge over some supervised models. These findings affirm the value of ZSP for validating event records and advancing ontology development. Our study underscores the efficacy of leveraging transfer learning and existing domain expertise to enhance research efficiency and scalability.