mlsanityData to 7 Oct 2026

AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

arXiv 2607.28617 ↗First posted 30 Jul 2026cs.AI cs.CL cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

32 of 43 references.

References · 32

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2020Extracting Training Data from Large Language Modelscited by 2,144
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2021Ethical and social risks of harm from Language Modelscited by 957
  5. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  6. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  7. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  8. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 159
  9. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  10. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 96
  11. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  12. 2023A Survey on Fairness in Large Language Modelscited by 91
20 more
  1. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  2. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safetycited by 71
  3. 2022Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governancecited by 67
  4. 2025Design Patterns for Securing LLM Agents against Prompt Injectionscited by 67
  5. 2023Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystemcited by 49
  6. 2024Dated Data: Tracing Knowledge Cutoffs in Large Language Modelscited by 48
  7. 2023Characterizing Manipulation from AI Systemscited by 43
  8. 2024AI auditing: The Broken Bus on the Road to AI Accountabilitycited by 42
  9. 2025Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducementscited by 31
  10. 2024Exploring the Personality Traits of LLMs through Latent Features Steeringcited by 21
  11. 2024From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditingcited by 15
  12. 2025Is Your LLM-Based Multi-Agent a Reliable Real-World Planner? Exploring Fraud Detection in Travel Planningcited by 14
  13. 2025WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AIcited by 13
  14. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Modelcited by 10
  15. 2026Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companiescited by 10
  16. 2024The Necessity of AI Audit Standards Boardscited by 9
  17. 2024Law and the Emerging Political Economy of Algorithmic Auditscited by 5
  18. 2026Who Controls the Conversation? User Perspectives On Generative AI (LLM) System Promptscited by 2
  19. 2023The right to audit and power asymmetries in algorithm auditingcited by 1
  20. 2025HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debatecited by 1

Built on by

Not cited yet.