# SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts

arXiv: https://arxiv.org/abs/2412.00765 · HTML: https://mlsanity.com/paper/2412.00765/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models](https://mlsanity.com/paper/2111.02840/) (2021)
- [Probing Neural Network Comprehension of Natural Language Arguments](https://mlsanity.com/paper/1907.07355/) (2019)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)
- [Adversarial Filters of Dataset Biases](https://mlsanity.com/paper/2002.04108/) (2020)
- [Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models](https://mlsanity.com/paper/2106.13353/) (2021)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Can Language Models be Biomedical Knowledge Bases?](https://mlsanity.com/paper/2109.07154/) (2021)
- [Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements](https://mlsanity.com/paper/2401.06766/) (2024)
- [GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective](https://mlsanity.com/paper/2211.08073/) (2022)
- [On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex](https://mlsanity.com/paper/2301.12868/) (2023)
- [Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning](https://mlsanity.com/paper/2310.13486/) (2023)
- [Examining the robustness of LLM evaluation to the distributional assumptions of benchmarks](https://mlsanity.com/paper/2404.16966/) (2024)
- [Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs](https://mlsanity.com/paper/2307.07312/) (2023)
- [A Survey on Out-of-Distribution Evaluation of Neural NLP Models](https://mlsanity.com/paper/2306.15261/) (2023)
- [Knowledge Questions from Knowledge Graphs](https://mlsanity.com/paper/1610.09935/) (2016)

## Built on by (most cited first, 10 of 10)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [RobustFlow: Towards Robust Agentic Workflow Generation](https://mlsanity.com/paper/2509.21834/) (2025)
- [PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximization](https://mlsanity.com/paper/2412.05892/) (2024)
- [PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usage](https://mlsanity.com/paper/2504.11509/) (2025)
- [A Reward-driven Automated Webshell Malicious-code Generator for Red-teaming](https://mlsanity.com/paper/2505.24252/) (2025)
- [Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions](https://mlsanity.com/paper/2607.12406/) (2026)
- [DMRL: Data- and Model-aware Reward Learning for Data Extraction](https://mlsanity.com/paper/2505.06284/) (2025)
- [ShellForge: Adversarial Co-Evolution of Webshell Generation and Multi-View Detection for Robust Webshell Defense](https://mlsanity.com/paper/2601.22182/) (2026)
- [Membership Inference for Contrastive Pre-training Models with Text-only PII Queries](https://mlsanity.com/paper/2603.14222/) (2026)
- [Let Prompts Bridge Defense Knowledge: Transferable Graph Purification via Vulnerability-Aware GPL](https://mlsanity.com/paper/2608.29054/) (2026)

## Abstract (arXiv)

Traditional methods for evaluating the robustness of large language models (LLMs) often rely on standardized benchmarks, which can escalate costs and limit evaluations across varied domains. This paper introduces a novel framework designed to autonomously evaluate the robustness of LLMs by incorporating refined adversarial prompts and domain-constrained knowledge guidelines in the form of knowledge graphs. Our method systematically generates descriptive sentences from domain-constrained knowledge graph triplets to formulate adversarial prompts, enhancing the relevance and challenge of the evaluation. These prompts, generated by the LLM itself and tailored to evaluate its own robustness, undergo a rigorous filtering and refinement process, ensuring that only those with high textual fluency and semantic fidelity are used. This self-evaluation mechanism allows the LLM to evaluate its robustness without the need for external benchmarks. We assess the effectiveness of our framework through extensive testing on both proprietary models like ChatGPT and open-source models such as Llama-3.1, Phi-3, and Mistral. Results confirm that our approach not only reduces dependency on conventional data but also provides a targeted and efficient means of evaluating LLM robustness in constrained domains.