mlsanityData to 7 Oct 2026

CyberPal.AI: Empowering LLMs with Expert-Driven Cybersecurity Instructions

arXiv 2408.09304 ↗First posted 17 Aug 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have significantly advanced natural language processing (NLP), providing versatile capabilities across various applications. However, their application to complex, domain-specific tasks, such as cyber-security, often faces substantial challenges. In this study, we introduce SecKnowledge and CyberPal.AI to address these challenges and train security-expert LLMs. SecKnowledge is a domain-knowledge-driven cyber-security instruction dataset, meticulously designed using years of accumulated expert knowledge in the domain through a multi-phase generation process. CyberPal.AI refers to a family of LLMs fine-tuned using SecKnowledge, aimed at building security-specialized LLMs capable of answering and following complex security-related instructions. Additionally, we introduce SecKnowledge-Eval, a comprehensive and diverse cyber-security evaluation benchmark, composed of an extensive set of cyber-security tasks we specifically developed to assess LLMs in the field of cyber-security, along with other publicly available security benchmarks. Our results show a significant average improvement of up to 24% over the baseline models, underscoring the benefits of our expert-driven instruction dataset generation process. These findings contribute to the advancement of AI-based cyber-security applications, paving the way for security-expert LLMs that can enhance threat-hunting and investigation processes.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 17 Aug 2024.

Builds on

29 of 45 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2023Mistral 7Bcited by 3,065
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
17 more
  1. 2024Mixtral of Expertscited by 1,785
  2. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  3. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  4. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  5. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  6. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  7. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  8. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  9. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  10. 2024CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligencecited by 61
  11. 2022SecureBERT: A Domain-Specific Language Model for Cybersecuritycited by 60
  12. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  13. 2023Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curationcited by 38
  14. 2023ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedbackcited by 34
  15. 2023Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistancecited by 33
  16. 2022CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domaincited by 31
  17. 2023Splitting the Difference on Adversarial Trainingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 6.

12 papers.

  1. 2025ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigationcited by 15
  2. 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
  3. 2024Toward Intelligent and Secure Cloud: Large Language Model Empowered Proactive Defensecited by 4
  4. 2025Toward Cybersecurity-Expert Small Language Modelscited by 3
  5. 2025Towards AI-Driven Human-Machine Co-Teaming for Adaptive and Agile Cyber Security Operation Centerscited by 2
  6. 2025False Alarms, Real Damage: Adversarial Attacks Using LLM-based Models on Text-based Cyber Threat Intelligence Systemscited by 2
  7. 2026Minerva: Reinforcement Learning with Verifiable Rewards for Cyber Threat Intelligence LLMscited by 2
  8. 2026To Defend Against Cyber Attacks, We Must Teach AI Agents to Hackcited by 1
  9. 2026MITRE-SAGE: A Multi-Agent Cybersecurity Question-Answering Modelcited by 0
  10. 2026From Threat Intelligence to Detection: Knowledge-driven Enrichment and Template-based Rule Grounding for Automated Sigma Rule Generationcited by 0
  11. 2026Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Makingcited by 0
  12. 2026MiST: Mid-Training LLMs for Cybersecuritycited by 0