mlsanityData to 7 Oct 2026

Black-Box Access is Insufficient for Rigorous AI Audits

arXiv 2401.14446 ↗First posted 25 Jan 2024cs.CY cs.AI cs.CRcited by 123

Not on the map. Why?

Abstract (arXiv)

External audits of AI systems are increasingly recognized as a key mechanism for AI governance. The effectiveness of an audit, however, depends on the degree of access granted to auditors. Recent audits of state-of-the-art AI systems have primarily relied on black-box access, in which auditors can only query the system and observe its outputs. However, white-box access to the system's inner workings (e.g., weights, activations, gradients) allows an auditor to perform stronger attacks, more thoroughly interpret models, and conduct fine-tuning. Meanwhile, outside-the-box access to training and deployment information (e.g., methodology, code, documentation, data, deployment details, findings from internal evaluations) allows auditors to scrutinize the development process and design more targeted evaluations. In this paper, we examine the limitations of black-box audits and the advantages of white- and outside-the-box audits. We also discuss technical, physical, and legal safeguards for performing these audits with minimal security risks. Given that different forms of access can lead to very different levels of evaluation, we conclude that (1) transparency regarding the access and methods used by auditors is necessary to properly interpret audit results, and (2) white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 204 of its 362 references that are in the map. First posted on arXiv 25 Jan 2024.

Builds on

204 of 362 references.

References · 204

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  12. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
38 more
  1. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  7. 2019Natural Adversarial Examplescited by 1,401
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  11. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  12. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  13. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  14. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  15. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  16. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  17. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  18. 2018Model Cards for Model Reportingcited by 1,068
  19. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  20. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  21. 2022Red Teaming Language Models with Language Modelscited by 983
  22. 2021Ethical and social risks of harm from Language Modelscited by 957
  23. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  24. 2023Steering Language Models With Activation Engineeringcited by 824
  25. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  26. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  27. 2023Whose Opinions Do Language Models Reflect?cited by 759
  28. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  29. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  30. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  31. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  32. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  33. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  34. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  35. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  36. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  37. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  38. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512

50 most cited of 204 All 204 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 46; 2025: 36; 2026: 41.

123 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  3. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  4. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  5. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  6. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  7. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  8. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  9. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  10. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 52
  11. 2024Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMscited by 52
  12. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 46
38 more
  1. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  2. 2024NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internalscited by 43
  3. 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 42
  4. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 33
  5. 2024Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementationcited by 27
  6. 2024Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generationcited by 25
  7. 2025DarkBench: Benchmarking Dark Patterns in Large Language Modelscited by 25
  8. 2025Mechanistic Interpretability of Emotion Inference in Large Language Modelscited by 24
  9. 2025Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisionscited by 23
  10. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18
  11. 2024From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditingcited by 15
  12. 2024SoK: Dataset Copyright Auditing in Machine Learning Systemscited by 14
  13. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 14
  14. 2024FairProof : Confidential and Certifiable Fairness for Neural Networkscited by 13
  15. 2024Near to Mid-term Risks and Opportunities of Open-Source Generative AIcited by 13
  16. 2024Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internalscited by 13
  17. 2024Training Compute Thresholds: Features and Functions in AI Regulationcited by 13
  18. 2024STAR: SocioTechnical Approach to Red Teaming Language Modelscited by 13
  19. 2025Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agendacited by 13
  20. 2025Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variationscited by 12
  21. 2024Risks and Opportunities of Open-Source Generative AIcited by 11
  22. 2024Position: An Inner Interpretability Framework for AI Inspired by Lessons from Cognitive Neurosciencecited by 11
  23. 2024Combining Large Language Models with Tutoring System Intelligence: A Case Study in Caregiver Homework Supportcited by 11
  24. 2024Responsible Reporting for Frontier AI Developmentcited by 10
  25. 2024Disguised Copyright Infringement of Latent Diffusion Modelscited by 10
  26. 2025Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culturecited by 10
  27. 2024Do Parameters Reveal More than Loss for Membership Inference?cited by 9
  28. 2024Verification methods for international AI agreementscited by 9
  29. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  30. 2024LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM statescited by 8
  31. 2025Discovering Forbidden Topics in Language Modelscited by 8
  32. 2025Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environmentscited by 8
  33. 2025Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustnesscited by 8
  34. 2024Addressing the regulatory gap: moving towards an EU AI audit ecosystem beyond the AI Act by including civil societycited by 7
  35. 2024Secure and Confidential Certificates of Online Fairnesscited by 7
  36. 2025The Accountability Paradox: How Platform API Restrictions Undermine AI Transparency Mandatescited by 7
  37. 2026Steered LLM Activations are Non-Surjectivecited by 7
  38. 2024Improving Intervention Efficacy via Concept Realignment in Concept Bottleneck Modelscited by 6

50 most cited of 123 All 123 (JSON)