mlsanityData to 7 Oct 2026

MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs

arXiv 2511.07107 ↗First posted 10 Nov 2025cs.AI cs.CLcited by 1

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Ensuring the safety of Large Language Models (LLMs) is critical for real-world deployment. However, current safety measures often fail to address implicit, domain-specific risks. To investigate this gap, we introduce a dataset of 3,000 annotated queries spanning education, finance, and management. Evaluations across 14 leading LLMs reveal a concerning vulnerability: an average jailbreak success rate of 57.8%. In response, we propose MENTOR, a metacognition-driven self-evolution framework. MENTOR performs metacognitive self-assessment, using strategies such as perspective-taking and consequential reasoning to uncover latent model misalignments. MENTOR couples single-pass rule-guided inference for routine requests with a selectively invoked metacognitive evolution cycle that revises residual unsafe responses, distills successful corrections into a dynamic rule graph, and compiles validated rules into activation-level steering signals for future inference. Experiments demonstrate that MENTOR substantially reduces attack success rates across all tested domains and outperforms existing safety alignment methods. The code and dataset for MENTOR are available at: https://anonymous.4open.science/r/MENTOR-Evo.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 32 of its 63 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

32 of 63 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025LLM-based Semantic Augmentation for Harmful Content Detection22 Apr
  2. 2024Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks23 Oct
  3. 2024Improving Alignment and Robustness with Circuit Breakers6 Jun
  4. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety8 Apr
  5. 2023Flames: Benchmarking Value Alignment of LLMs in Chinese12 Nov

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  10. 2023Steering Language Models With Activation Engineeringcited by 824
  11. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  12. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
15 more
  1. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  2. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 235
  3. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  4. 2025SEAL: Steerable Reasoning Calibration of Large Language Models for Freecited by 65
  5. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  6. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attackscited by 35
  7. 2024What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attackscited by 24
  8. 2025EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarioscited by 16
  9. 2025MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Applicationcited by 14
  10. 2025C3AI: Crafting and Evaluating Constitutions for Constitutional AIcited by 13
  11. 2025SAKE: Steering Activations for Knowledge Editingcited by 12
  12. 2024RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Processcited by 10
  13. 2025Improving Large Language Model Safety with Contrastive Representation Learningcited by 7
  14. 2024Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Modelscited by 4
  15. 2025Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Beyond the Illusion of Consensus: From Surface Heuristics to Knowledge-Grounded Evaluation in LLM-as-a-Judgecited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning
  2. 2026Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization
  3. 2026Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
  4. 2024Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM
  5. 2025Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization
  6. 2026Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
  7. 2026Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
  8. 2025Picky LLMs and Unreliable RMs: An Empirical Study on Safety Alignment after Instruction Tuning