mlsanityData to 14 Oct 2026

Can We Trust AI to Govern AI? Benchmarking LLM Performance on Privacy and AI Governance Exams

arXiv 2508.09036 ↗First posted 12 Aug 2025cs.CY cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

The rapid emergence of large language models (LLMs) has raised urgent questions across the modern workforce about this new technology's strengths, weaknesses, and capabilities. For privacy professionals, the question is whether these AI systems can provide reliable support on regulatory compliance, privacy program management, and AI governance. In this study, we evaluate ten leading open and closed LLMs, including models from OpenAI, Anthropic, Google DeepMind, Meta, and DeepSeek, by benchmarking their performance on industry-standard certification exams: CIPP/US, CIPM, CIPT, and AIGP from the International Association of Privacy Professionals (IAPP). Each model was tested using official sample exams in a closed-book setting and compared to IAPP's passing thresholds. Our findings show that several frontier models such as Gemini 2.5 Pro and OpenAI's GPT-5 consistently achieve scores exceeding the standards for professional human certification - demonstrating substantial expertise in privacy law, technical controls, and AI governance. The results highlight both the strengths and domain-specific gaps of current LLMs and offer practical insights for privacy officers, compliance leads, and technologists assessing the readiness of AI tools for high-stakes data governance roles. This paper provides an overview for professionals navigating the intersection of AI advancement and regulatory risk and establishes a machine benchmark based on human-centric evaluations.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 24 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

6 of 24 references.

References · 6

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  6. 2025Consumer Beware! Exploring Data Brokers' CCPA Compliancecited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.