# Benchmarking LLMs for Political Science: A United Nations Perspective

arXiv: https://arxiv.org/abs/2502.14122 · HTML: https://mlsanity.com/paper/2502.14122/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 19 Feb 2025.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [BillSum: A Corpus for Automatic Summarization of US Legislation](https://mlsanity.com/paper/1910.00523/) (2019)
- [LawLLM: Law Large Language Model for the US Legal System](https://mlsanity.com/paper/2407.21065/) (2024)
- [Author's Sentiment Prediction](https://mlsanity.com/paper/2011.06128/) (2020)
- [Public Wisdom Matters! Discourse-Aware Hyperbolic Fourier Co-Attention for Social-Text Classification](https://mlsanity.com/paper/2209.13017/) (2022)

## Built on by (most cited first, 10 of 10)

- [Replicating Human Motivated Reasoning Studies with LLMs](https://mlsanity.com/paper/2601.16130/) (2026)
- [PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives](https://mlsanity.com/paper/2505.19558/) (2025)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- ["As Eastern Powers, I will veto." : An Investigation of Nation-level Bias of Large Language Models in International Relations](https://mlsanity.com/paper/2511.10695/) (2025)
- [When AI Navigates the Fog of War](https://mlsanity.com/paper/2603.16642/) (2026)
- [PolicyLLM: Towards Excellent Comprehension of Public Policy for Large Language Models](https://mlsanity.com/paper/2604.12995/) (2026)
- [BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces](https://mlsanity.com/paper/2606.02798/) (2026)
- [The Foreign Policy AI Evaluation Gap](https://mlsanity.com/paper/2607.02955/) (2026)
- [Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts](https://mlsanity.com/paper/2608.06123/) (2026)
- [TradeVerse: A Longitudinal Benchmark of Political Negotiation in International Trade](https://mlsanity.com/paper/2608.06549/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have achieved significant advances in natural language processing, yet their potential for high-stake political decision-making remains largely unexplored. This paper addresses the gap by focusing on the application of LLMs to the United Nations (UN) decision-making process, where the stakes are particularly high and political decisions can have far-reaching consequences. We introduce a novel dataset comprising publicly available UN Security Council (UNSC) records from 1994 to 2024, including draft resolutions, voting records, and diplomatic speeches. Using this dataset, we propose the United Nations Benchmark (UNBench), the first comprehensive benchmark designed to evaluate LLMs across four interconnected political science tasks: co-penholder judgment, representative voting simulation, draft adoption prediction, and representative statement generation. These tasks span the three stages of the UN decision-making process--drafting, voting, and discussing--and aim to assess LLMs' ability to understand and simulate political dynamics. Our experimental analysis demonstrates the potential and challenges of applying LLMs in this domain, providing insights into their strengths and limitations in political science. This work contributes to the growing intersection of AI and political science, opening new avenues for research and practical applications in global governance. The UNBench Repository can be accessed at: https://github.com/yueqingliang1/UNBench.