# Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark

arXiv: https://arxiv.org/abs/2508.05674 · HTML: https://mlsanity.com/paper/2508.05674/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 11 of 11)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security](https://mlsanity.com/paper/2406.05590/) (2024)
- [An Empirical Evaluation of LLMs for Solving Offensive Security Challenges](https://mlsanity.com/paper/2402.11814/) (2024)
- [Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questions](https://mlsanity.com/paper/2308.10443/) (2023)
- [D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System with Planner and Heterogeneous Executors for Offensive Security](https://mlsanity.com/paper/2502.10931/) (2025)
- [CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Execution](https://mlsanity.com/paper/2505.17107/) (2025)
- [Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applications](https://mlsanity.com/paper/2504.02867/) (2025)

## Built on by (most cited first, 12 of 12)

- [Security in LLM-as-a-Judge: A Comprehensive SoK](https://mlsanity.com/paper/2603.29403/) (2026)
- [CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking](https://mlsanity.com/paper/2602.08023/) (2026)
- [AI In Cybersecurity Education -- Scalable Agentic CTF Design Principles and Educational Outcomes](https://mlsanity.com/paper/2603.21551/) (2026)
- [Autonomous Adversary: Red-Teaming in the age of LLM](https://mlsanity.com/paper/2605.06486/) (2026)
- [LLMs for Secure Hardware Design and Related Problems: Opportunities and Challenges](https://mlsanity.com/paper/2605.10807/) (2026)
- [Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response](https://mlsanity.com/paper/2607.25379/) (2026)
- [The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark](https://mlsanity.com/paper/2608.11469/) (2026)
- [RAVEN: Retrieval-Augmented Vulnerability Exploration Network for Memory Corruption Analysis in User Code and Binary Programs](https://mlsanity.com/paper/2604.17948/) (2026)
- [Determinants and Limits of LLM Security-Tool Orchestration: A Study with HexStrike-AI](https://mlsanity.com/paper/2607.02873/) (2026)
- [Semi-Automated Detection of Gaps in LLM Security Knowledge](https://mlsanity.com/paper/2607.18496/) (2026)
- [Reveree: Diagnosing LLM Reverse-Engineering Agents](https://mlsanity.com/paper/2609.01185/) (2026)
- [Humans vs. Machines: A Comparison With AI Agents in Software Reverse Engineering](https://mlsanity.com/paper/2610.14960/) (2026)

## Abstract (arXiv)

Recent advances in LLM agentic systems have improved the automation of offensive security tasks, particularly for Capture the Flag (CTF) challenges. We systematically investigate the key factors that drive agent success and provide a detailed recipe for building effective LLM-based offensive security agents. First, we present CTFJudge, a framework leveraging LLM as a judge to analyze agent trajectories and provide granular evaluation across CTF solving steps. Second, we propose a novel metric, CTF Competency Index (CCI) for partial correctness, revealing how closely agent solutions align with human-crafted gold standards. Third, we examine how LLM hyperparameters, namely temperature, top-p, and maximum token length, influence agent performance and automated cybersecurity task planning. For rapid evaluation, we present CTFTiny, a curated benchmark of 50 representative CTF challenges across binary exploitation, web, reverse engineering, forensics, and cryptography. Our findings identify optimal multi-agent coordination settings and lay the groundwork for future LLM agent research in cybersecurity. We make CTFTiny open source to public https://github.com/NYU-LLM-CTF/CTFTiny along with CTFJudge on https://github.com/NYU-LLM-CTF/CTFJudge.