# Large Language Model-Based Reward Design for Deep Reinforcement Learning-Driven Autonomous Cyber Defense

arXiv: https://arxiv.org/abs/2511.16483 · HTML: https://mlsanity.com/paper/2511.16483/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 20 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Large Language Models are Autonomous Cyber Defenders](https://mlsanity.com/paper/2505.04843/) (2025)
- [Constraints Satisfiability Driven Reinforcement Learning for Autonomous Cyber Defense](https://mlsanity.com/paper/2104.08994/) (2021)
- [Deep Reinforcement Learning for Cyber System Defense under Dynamic Adversarial Uncertainties](https://mlsanity.com/paper/2302.01595/) (2023)
- [L2M-AID: Autonomous Cyber-Physical Defense by Fusing Semantic Reasoning of Large Language Models with Multi-Agent Reinforcement Learning (Preprint)](https://mlsanity.com/paper/2510.07363/) (2025)

## Built on by (most cited first, 1 of 1)

- [Towards Hierarchical Cyber Defense with Large Language Models: From Planning to Execution](https://mlsanity.com/paper/2610.00590/) (2026)

## Abstract (arXiv)

Designing rewards for autonomous cyber attack and defense learning agents in a complex, dynamic environment is a challenging task for subject matter experts. We propose a large language model (LLM)-based reward design approach to generate autonomous cyber defense policies in a deep reinforcement learning (DRL)-driven experimental simulation environment. Multiple attack and defense agent personas were crafted, reflecting heterogeneity in agent actions, to generate LLM-guided reward designs where the LLM was first provided with contextual cyber simulation environment information. These reward structures were then utilized within a DRL-driven attack-defense simulation environment to learn an ensemble of cyber defense policies. Our results suggest that LLM-guided reward designs can lead to effective defense strategies against diverse adversarial behaviors.