# Augmenting Greybox Fuzzing with Generative AI

arXiv: https://arxiv.org/abs/2306.06782 · HTML: https://mlsanity.com/paper/2306.06782/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 11 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [Jigsaw: Large Language Models meet Program Synthesis](https://mlsanity.com/paper/2112.02969/) (2021)
- [Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code](https://mlsanity.com/paper/2206.01335/) (2022)
- [Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT](https://mlsanity.com/paper/2304.02014/) (2023)
- [FairFuzz: Targeting Rare Branches to Rapidly Increase Greybox Fuzz Testing Coverage](https://mlsanity.com/paper/1709.07101/) (2017)
- [NEUZZ: Efficient Fuzzing with Neural Program Smoothing](https://mlsanity.com/paper/1807.05620/) (2018)
- [Superion: Grammar-Aware Greybox Fuzzing](https://mlsanity.com/paper/1812.01197/) (2018)
- [MTFuzz: Fuzzing with a Multi-Task Neural Network](https://mlsanity.com/paper/2005.12392/) (2020)
- [UNIFUZZ: A Holistic and Pragmatic Metrics-Driven Platform for Evaluating Fuzzers](https://mlsanity.com/paper/2010.01785/) (2020)
- [Building Fast Fuzzers](https://mlsanity.com/paper/1911.07707/) (2019)
- [Inferring Input Grammars from Dynamic Control Flow](https://mlsanity.com/paper/1912.05937/) (2019)

## Built on by (most cited first, 18 of 18)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [KernelGPT: Enhanced Kernel Fuzzing via Large Language Models](https://mlsanity.com/paper/2401.00563/) (2023)
- [LLAMAFUZZ: Large Language Model Enhanced Greybox Fuzzing](https://mlsanity.com/paper/2406.07714/) (2024)
- [A Contemporary Survey of Large Language Model Assisted Program Analysis](https://mlsanity.com/paper/2502.18474/) (2025)
- [Locus: Agentic Predicate Synthesis for Directed Fuzzing](https://mlsanity.com/paper/2508.21302/) (2025)
- [`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMs](https://mlsanity.com/paper/2502.00735/) (2025)
- [Exploring the Limits of ChatGPT in Software Security Applications](https://mlsanity.com/paper/2312.05275/) (2023)
- [On the Challenges of Fuzzing Techniques via Large Language Models](https://mlsanity.com/paper/2402.00350/) (2024)
- [Retrieval Augmented Generation Based LLM Evaluation For Protocol State Machine Inference With Chain-of-Thought Reasoning](https://mlsanity.com/paper/2502.15727/) (2025)
- [Generative transformations and patterns in LLM-native approaches for software verification and falsification](https://mlsanity.com/paper/2404.09384/) (2024)
- [MigGPT: Harnessing Large Language Models for Automated Migration of Out-of-Tree Linux Kernel Patches Across Versions](https://mlsanity.com/paper/2504.09474/) (2025)
- [Towards Reliable LLM-Driven Fuzz Testing: Vision and Road Ahead](https://mlsanity.com/paper/2503.00795/) (2025)
- [An Agentic Flow for Finite State Machine Extraction using Prompt Chaining](https://mlsanity.com/paper/2507.11222/) (2025)
- [MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzing](https://mlsanity.com/paper/2508.14300/) (2025)
- [LLMs are All You Need? Improving Fuzz Testing for MOJO with Large Language Models](https://mlsanity.com/paper/2510.10179/) (2025)

## Abstract (arXiv)

Real-world programs expecting structured inputs often has a format-parsing stage gating the deeper program space. Neither a mutation-based approach nor a generative approach can provide a solution that is effective and scalable. Large language models (LLM) pre-trained with an enormous amount of natural language corpus have proved to be effective for understanding the implicit format syntax and generating format-conforming inputs. In this paper, propose ChatFuzz, a greybox fuzzer augmented by generative AI. More specifically, we pick a seed in the fuzzer's seed pool and prompt ChatGPT generative models to variations, which are more likely to be format-conforming and thus of high quality. We conduct extensive experiments to explore the best practice for harvesting the power of generative LLM models. The experiment results show that our approach improves the edge coverage by 12.77% over the SOTA greybox fuzzer (AFL++) on 12 target programs from three well-tested benchmarks. As for vulnerability detection, sys is able to perform similar to or better than AFL++ for programs with explicit syntax rules but not for programs with non-trivial syntax.