# LLAMAFUZZ: Large Language Model Enhanced Greybox Fuzzing

arXiv: https://arxiv.org/abs/2406.07714 · HTML: https://mlsanity.com/paper/2406.07714/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 78 references that are in the map.
- First posted on arXiv 11 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models](https://mlsanity.com/paper/2212.14834/) (2022)
- [WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models](https://mlsanity.com/paper/2310.15991/) (2023)
- [Evaluating Fuzz Testing](https://mlsanity.com/paper/1808.09700/) (2018)
- [Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source](https://mlsanity.com/paper/2201.06589/) (2022)
- [FairFuzz: Targeting Rare Branches to Rapidly Increase Greybox Fuzz Testing Coverage](https://mlsanity.com/paper/1709.07101/) (2017)
- [Augmenting Greybox Fuzzing with Generative AI](https://mlsanity.com/paper/2306.06782/) (2023)
- [Magma: A Ground-Truth Fuzzing Benchmark](https://mlsanity.com/paper/2009.01120/) (2020)
- [Superion: Grammar-Aware Greybox Fuzzing](https://mlsanity.com/paper/1812.01197/) (2018)
- [Smart Greybox Fuzzing](https://mlsanity.com/paper/1811.09447/) (2018)
- [Snipuzz: Black-box Fuzzing of IoT Firmware via Message Snippet Inference](https://mlsanity.com/paper/2105.05445/) (2021)
- [Compressed-Language Models for Understanding Compressed File Formats: a JPEG Exploration](https://mlsanity.com/paper/2405.17146/) (2024)
- [SmartSeed: Smart Seed Generation for Efficient Fuzzing](https://mlsanity.com/paper/1807.02606/) (2018)
- [CovRL: Fuzzing JavaScript Engines with Coverage-Guided Reinforcement Learning for LLM-based Mutation](https://mlsanity.com/paper/2402.12222/) (2024)
- [Advanced White-Box Heuristics for Search-Based Fuzzing of REST APIs](https://mlsanity.com/paper/2309.08360/) (2023)
- [WEIZZ: Automatic Grey-box Fuzzing for Structured Binary Formats](https://mlsanity.com/paper/1911.00621/) (2019)
- [Hyperfuzzing: black-box security hypertesting with a grey-box fuzzer](https://mlsanity.com/paper/2308.09081/) (2023)
- [WhisperFuzz: White-Box Fuzzing for Detecting and Locating Timing Vulnerabilities in Processors](https://mlsanity.com/paper/2402.03704/) (2024)

## Built on by (most cited first, 14 of 14)

- [Towards Resilient and Efficient LLMs: A Comparative Study of Efficiency, Performance, and Adversarial Robustness](https://mlsanity.com/paper/2408.04585/) (2024)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [A Contemporary Survey of Large Language Model Assisted Program Analysis](https://mlsanity.com/paper/2502.18474/) (2025)
- [Hybrid Fuzzing with LLM-Guided Input Mutation and Semantic Feedback](https://mlsanity.com/paper/2511.03995/) (2025)
- [PBFuzz: Agentic Directed Fuzzing for PoV Generation](https://mlsanity.com/paper/2512.04611/) (2025)
- [FuzzAug: Data Augmentation by Coverage-guided Fuzzing for Neural Test Generation](https://mlsanity.com/paper/2406.08665/) (2024)
- [Adversarial Agent Collaboration for Correctness Improvements of C to Safe Rust Translation](https://mlsanity.com/paper/2510.03879/) (2025)
- [Evaluating Program Semantics Reasoning with Type Inference in System F](https://mlsanity.com/paper/2509.23686/) (2025)
- [Code Generation by Differential Test Time Scaling](https://mlsanity.com/paper/2605.20473/) (2026)
- [Content Fuzzing for Escaping Information Cocoons on Digital Social Media](https://mlsanity.com/paper/2604.05461/) (2026)
- [FunFuzz: An LLM-Powered Evolutionary Fuzzing Framework](https://mlsanity.com/paper/2605.02789/) (2026)
- [The Greatness of Science Cannot Be Planned: Agentic Auto-Research is Fuzz Testing](https://mlsanity.com/paper/2608.09855/) (2026)
- [Semantic-Aware Fuzzing: An Empirical Framework for LLM-Guided, Reasoning-Driven Input Mutation](https://mlsanity.com/paper/2509.19533/) (2025)
- [SyzHarness: Patch-Based Kernel Bug Reproduction with LLM-Synthesized Fuzzing Harnesses](https://mlsanity.com/paper/2609.23889/) (2026)

## Abstract (arXiv)

Greybox fuzzing has achieved success in revealing bugs and vulnerabilities in programs. However, randomized mutation strategies have limited the fuzzer's performance on structured data. Specialized fuzzers can handle complex structured data, but require additional efforts in grammar and suffer from low throughput. In this paper, we explore the potential of utilizing the Large Language Model to enhance greybox fuzzing for structured data. We utilize the pre-trained knowledge of LLM about data conversion and format to generate new valid inputs. We further fine-tuned it with paired mutation seeds to learn structured format and mutation strategies effectively. Our LLM-based fuzzer, LLAMAFUZZ, integrates the power of LLM to understand and mutate structured data to fuzzing. We conduct experiments on the standard bug-based benchmark Magma and a wide variety of real-world programs. LLAMAFUZZ outperforms our top competitor by 41 bugs on average. We also identified 47 unique bugs across all trials. Moreover, LLAMAFUZZ demonstrated consistent performance on both bug trigger and bug reached. Compared to AFL++, LLAMAFUZZ achieved 27.19% more branches in real-world program sets on average. We also demonstrate a case study to explain how LLMs enhance the fuzzing process in terms of code coverage.