mlsanityData to 14 Oct 2026

Real-World Gaps in AI Governance Research

arXiv 2505.00174 ↗First posted 30 Apr 2025cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Drawing on 1,178 safety and reliability papers from 9,439 generative AI papers (January 2020 - March 2025), we compare research outputs of leading AI companies (Anthropic, Google DeepMind, Meta, Microsoft, and OpenAI) and AI universities (CMU, MIT, NYU, Stanford, UC Berkeley, and University of Washington). We find that corporate AI research increasingly concentrates on pre-deployment areas -- model alignment and testing & evaluation -- while attention to deployment-stage issues such as model bias has waned. Significant research gaps exist in high-risk deployment domains, including healthcare, finance, misinformation, persuasive and addictive features, hallucinations, and copyright. Without improved observability into deployed AI, growing corporate concentration could deepen knowledge deficits. We recommend expanding external researcher access to deployment data and systematic observability of in-market AI behaviors.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 43 of its 120 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

43 of 120 references.

References · 43

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2020Learning to summarize from human feedbackcited by 2,588
  6. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  7. 2021Ethical and social risks of harm from Language Modelscited by 966
  8. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  9. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  10. 2019Release Strategies and the Social Impacts of Language Modelscited by 513
  11. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  12. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
31 more
  1. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  2. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
  3. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  4. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  5. 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 98
  6. 2022The Fallacy of AI Functionalitycited by 88
  7. 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 72
  8. 2020The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Researchcited by 67
  9. 2025Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMscited by 54
  10. 2025Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactionscited by 52
  11. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 47
  12. 2025Toward an Evaluation Science for Generative AI Systemscited by 45
  13. 2020The Grey Hoodie Project: Big Tobacco, Big Tech, and the threat on academic integritycited by 38
  14. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 37
  15. 2024HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wildcited by 27
  16. 2024FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluationcited by 25
  17. 2025Lessons From Red Teaming 100 Generative AI Productscited by 23
  18. 2024Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Datacited by 20
  19. 2024Holistic Safety and Responsibility Evaluations of Advanced AI Modelscited by 19
  20. 2019Regulatory Markets for AI Safetycited by 13
  21. 2024Scaling Up LLM Reviews for Google Ads Content Moderationcited by 12
  22. 2023BiasX: "Thinking Slow" in Toxic Content Moderation with Explanations of Implied Social Biasescited by 7
  23. 2023Analyzing the Impact of Companies on AI Research Based on Publicationscited by 5
  24. 2023Who is leading in AI? An analysis of industry AI researchcited by 4
  25. 2024The complementary contributions of academia and industry to AI researchcited by 4
  26. 2024Mapping Technical Safety Research at AI Companies: A literature review and incentives analysiscited by 4
  27. 2024Assessing Bias in Metric Models for LLM Open-Ended Generation Bias Benchmarkscited by 4
  28. 2024The Narrow Depth and Breadth of Corporate Responsible AI Researchcited by 2
  29. 2024Zero-Shot Image Moderation in Google Ads with LLM-Assisted Textual Descriptions and Cross-modal Co-embeddingscited by 2
  30. 2023Nip it in the Bud: Moderation Strategies in Open Source Software Projects and the Role of Botscited by 1
  31. 2025Enabling External Scrutiny of AI Systems with Privacy-Enhancing Technologiescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effectscited by 19
  2. 2025Statistical Methods in Generative AIcited by 4
  3. 2025A Maslow-Inspired Hierarchy of Engagement with AI Modelcited by 0
  4. 2026Studying People to Study AI: Expert Perspectives on the Epistemic Fit and Barriers of Human Research in AI Safety & Ethicscited by 0
  5. 2026Context-Aware Pre-Deployment Evaluation of AI Systems: A Regulatory Framework for Nigerian Fintechcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Agentic AI in Industry: Adoption Level and Deployment Barriers
  2. 2025Documenting Deployment with Fabric: A Repository of Real-World AI Governance
  3. 2025Who is Responsible? The Data, Models, Users or Regulations? A Comprehensive Survey on Responsible Generative AI for a Sustainable Future
  4. 2026Governing the Agentic Enterprise: A Governance Maturity Model for Managing AI Agent Sprawl in Business Operations
  5. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap
  6. 2025A Different Approach to AI Safety: Proceedings from the Columbia Convening on Openness in Artificial Intelligence and AI Safety
  7. 2026From Replacement to Orchestration: A Socio-Technical Architecture for Agentic AI in Corporate R&D
  8. 2026The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI