# Benchmarking Open-Weight Foundation Models for Global AI Technical Governance

arXiv: https://arxiv.org/abs/2606.26099 · HTML: https://mlsanity.com/paper/2606.26099/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 12 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [Dated Data: Tracing Knowledge Cutoffs in Large Language Models](https://mlsanity.com/paper/2403.12958/) (2024)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://mlsanity.com/paper/2402.10946/) (2024)
- [Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research](https://mlsanity.com/paper/2412.04497/) (2024)
- [The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities](https://mlsanity.com/paper/2508.05525/) (2025)
- [LLMs Can Infer Political Alignment from Online Conversations](https://mlsanity.com/paper/2603.11253/) (2026)
- [Can Large Language Models Become Policy Refinement Partners? Evidence from China's Social Security Studies](https://mlsanity.com/paper/2504.09137/) (2025)
- [Global AI Bias Audit for Technical Governance](https://mlsanity.com/paper/2602.13246/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly deployed in artificial intelligence (AI) governance analysis across national and international organisations. There is, however, growing evidence that such models produce significantly less accurate responses for countries that are underrepresented in their training data-a pattern described in existing literature as geographic bias. Existing studies examining this phenomenon are subject to three methodological limitations that together undermine their findings: (1) reliance on proprietary systems whose weights are not publicly released, which prevents independent replication; (2) evaluation of model knowledge about years that fall after data collection for model training had concluded, leading to geographic ignorance in addition to the natural limits of each model's knowledge; and (3) use of coarse binary response classification that cannot distinguish models' confident fabrication (HF) from their honest acknowledgement of uncertainty. This study addresses all three limitations by benchmarking four open-weight frontier language models against the Global AI Dataset v2 (GAID v2), a verified ground-truth database of 24,453 indicators across 227 countries published on Harvard Dataverse in January 2026. A total of 18 indicators, mapped to the eight thematic dimensions of the IEEE IRAI 2026 framework, are selected from GAID v2, yielding approximately 2,990 country-metric-year observations across six evaluation years (within the period of 2010-2023). Model responses are classified using a five-category scheme that distinguishes (a) verified accuracy (VA), (b) HF, (c) honest refusal (HR), (d) qualitative hedging (QH), and (e) misattribution (MF). Geographic disparities in accuracy are estimated through mixed-effects logistic regression and difference-in-differences (DiD) analysis.