mlsanityData to 13 Oct 2026

Open Problems in Technical AI Governance

arXiv 2407.14981 ↗First posted 20 Jul 2024cs.CYcited by 49

Not on the map. Why?

Abstract (arXiv)

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

Key facts. Cited by 49 papers in the map (data through 13 Oct 2026). First posted on arXiv 20 Jul 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 12; 2025: 15; 2026: 22.

49 papers.

Sort:

By area:

  1. 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
  2. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 87
  3. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 71
  4. 2025Authenticated Delegation and Authorized AI Agentscited by 63
  5. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  6. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 43
  7. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 37
  8. 2025Adversarial Attacks on Robotic Vision Language Action Modelscited by 31
  9. 2025Adversarial Reasoning at Jailbreaking Timecited by 28
  10. 2024Can Editing LLMs Inject Harm?cited by 22
  11. 2024Measuring Free-Form Decision-Making Inconsistency of Language Models in Military Crisis Simulationscited by 11
  12. 2025CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuningcited by 11
37 more
  1. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  2. 2024The Reality of AI and Bioriskcited by 8
  3. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 7
  4. 2024Nob-MIAs: Non-biased Membership Inference Attacks Assessment on Large Language Models with Ex-Post Dataset Constructioncited by 5
  5. 2024Jailbreaking LLM-Controlled Robotscited by 5
  6. 2025A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Searchcited by 5
  7. 2025MoralReason: Generalizable Moral Decision Alignment For LLM Agents Using Reasoning-Level Reinforcement Learningcited by 5
  8. 2024Towards Data Governance of Frontier AI Modelscited by 3
  9. 2025Practical Principles for AI Cost and Compute Accountingcited by 3
  10. 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 3
  11. 2026FLIPS: Instance-Fingerprinting for LLMs via Pseudo-random Sequencescited by 3
  12. 2025AI Agents Should be Regulated Based on the Extent of Their Autonomous Operationscited by 2
  13. 2026Quantifying the Effect of Test Set Contamination on Generative Evaluationscited by 2
  14. 2026How Does "English (US)" Become the Default? Triangulating Structural Bias Towards American English Across the LLM Pipelinecited by 2
  15. 2026Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demandscited by 2
  16. 2024Shaping AI's Impact on Billions of Livescited by 1
  17. 2024Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluationscited by 1
  18. 2025LibVulnWatch: A Deep Assessment Agent System and Leaderboard for Uncovering Hidden Vulnerabilities in Open-Source AI Librariescited by 1
  19. 2025A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanationscited by 1
  20. 2025Societal Capacity Assessment Framework: Measuring Resilience to Inform Advanced AI Risk Managementcited by 1
  21. 2026SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testingcited by 1
  22. 2025Dimensional Characterization and Pathway Modeling for Catastrophic AI Riskscited by 0
  23. 2026The Limits of AI Data Transparency Policy: Three Disclosure Fallaciescited by 0
  24. 2026Distributed Legal Infrastructure for a Trustworthy Agentic Webcited by 0
  25. 2026Designing escalation criteria for international AI incident response: criteria, triggers, and thresholdscited by 0
  26. 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocolcited by 0
  27. 2026Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routingcited by 0
  28. 2026AI Sandboxes: A Threat Model, Taxonomy, and Measurement Frameworkcited by 0
  29. 2026Exposure is not manifestation: measurement target and output resolution jointly determine which behavioural-faithfulness evaluator winscited by 0
  30. 2026Privacy-Preserving AI Verification via Minimal Information Disclosurecited by 0
  31. 2026Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the Modelcited by 0
  32. 2026Workload Identification with Physical Side Channels for AI Governancecited by 0
  33. 2026Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governancecited by 0
  34. 2026Hard-Gate Candidacy in a Deployed Validator Suitecited by 0
  35. 2026Discriminating Fixture Coverage in Agent-Infrastructure Verification Suitescited by 0
  36. 2026Can Power Draw Constrain Covert Compute? Limits of Analogue Verification for AI Governancecited by 0
  37. 2026SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routingcited by 0