# Open Problems in Technical AI Governance

arXiv: https://arxiv.org/abs/2407.14981 · HTML: https://mlsanity.com/paper/2407.14981/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2024.

## Built on by (most cited first, 30 of 49)

- [Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation](https://mlsanity.com/paper/2409.01586/) (2024)
- [Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey](https://mlsanity.com/paper/2409.18169/) (2024)
- [Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning](https://mlsanity.com/paper/2408.09600/) (2024)
- [Authenticated Delegation and Authorized AI Agents](https://mlsanity.com/paper/2501.09674/) (2025)
- [Knowledge Mechanisms in Large Language Models: A Survey and Perspective](https://mlsanity.com/paper/2407.15017/) (2024)
- [The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It](https://mlsanity.com/paper/2505.24119/) (2025)
- [Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities](https://mlsanity.com/paper/2502.05209/) (2025)
- [Adversarial Attacks on Robotic Vision Language Action Models](https://mlsanity.com/paper/2506.03350/) (2025)
- [Adversarial Reasoning at Jailbreaking Time](https://mlsanity.com/paper/2502.01633/) (2025)
- [Can Editing LLMs Inject Harm?](https://mlsanity.com/paper/2407.20224/) (2024)
- [Measuring Free-Form Decision-Making Inconsistency of Language Models in Military Crisis Simulations](https://mlsanity.com/paper/2410.13204/) (2024)
- [CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning](https://mlsanity.com/paper/2505.16559/) (2025)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [The Reality of AI and Biorisk](https://mlsanity.com/paper/2412.01946/) (2024)
- [Jailbreaking LLM-Controlled Robots](https://mlsanity.com/paper/2410.13691/) (2024)
- [Nob-MIAs: Non-biased Membership Inference Attacks Assessment on Large Language Models with Ex-Post Dataset Construction](https://mlsanity.com/paper/2408.05968/) (2024)
- [A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Search](https://mlsanity.com/paper/2507.00004/) (2025)
- [MoralReason: Generalizable Moral Decision Alignment For LLM Agents Using Reasoning-Level Reinforcement Learning](https://mlsanity.com/paper/2511.12271/) (2025)
- [Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sink](https://mlsanity.com/paper/2602.05228/) (2026)
- [Towards Data Governance of Frontier AI Models](https://mlsanity.com/paper/2412.03824/) (2024)
- [Practical Principles for AI Cost and Compute Accounting](https://mlsanity.com/paper/2502.15873/) (2025)
- [How Does "English (US)" Become the Default? Triangulating Structural Bias Towards American English Across the LLM Pipeline](https://mlsanity.com/paper/2604.04204/) (2026)
- [FLIPS: Instance-Fingerprinting for LLMs via Pseudo-random Sequences](https://mlsanity.com/paper/2606.03330/) (2026)
- [AI Agents Should be Regulated Based on the Extent of Their Autonomous Operations](https://mlsanity.com/paper/2503.04750/) (2025)
- [Quantifying the Effect of Test Set Contamination on Generative Evaluations](https://mlsanity.com/paper/2601.04301/) (2026)
- [Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands](https://mlsanity.com/paper/2605.15164/) (2026)
- [Shaping AI's Impact on Billions of Lives](https://mlsanity.com/paper/2412.02730/) (2024)
- [Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations](https://mlsanity.com/paper/2412.15433/) (2024)
- [LibVulnWatch: A Deep Assessment Agent System and Leaderboard for Uncovering Hidden Vulnerabilities in Open-Source AI Libraries](https://mlsanity.com/paper/2505.08842/) (2025)

## Abstract (arXiv)

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.