mlsanityData to 14 Oct 2026

WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models

arXiv 2310.15991 ↗First posted 24 Oct 2023cs.SE cs.LG cs.PLcited by 37

Not on the map. Why?

Abstract (arXiv)

Compiler correctness is crucial, as miscompilation can falsify program behaviors, leading to serious consequences. Fuzzing has been studied to uncover compiler defects. However, compiler fuzzing remains challenging: Existing arts focus on black- and grey-box fuzzing, which generates tests without sufficient understanding of internal compiler behaviors. Meanwhile, traditional white-box techniques, like symbolic execution, are computationally inapplicable to the giant codebase of compilers. Recent advances demonstrate that Large Language Models (LLMs) excel in code generation/understanding tasks. Nonetheless, guiding LLMs with compiler source-code information remains a missing piece of research in compiler testing. To this end, we propose WhiteFox, the first white-box compiler fuzzer using LLMs with source-code information to test compiler optimization, with a spotlight on detecting deep logic bugs in the deep learning (DL) compilers. WhiteFox adopts a multi-agent framework: an LLM-based analysis agent examines the low-level optimization source code and produces requirements on the high-level test programs that can trigger the optimization; an LLM-based generation agent produces test programs based on the summarized requirements. Additionally, optimization-triggering tests are used as feedback to enhance the generation on the fly. Our evaluation on the three most popular DL compilers (i.e., PyTorch Inductor, TensorFlow-XLA, and TensorFlow Lite) shows WhiteFox can generate high-quality test programs to exercise deep optimizations, practicing up to 8X more than state-of-the-art fuzzers. WhiteFox has found 101 bugs for the DL compilers, with 92 confirmed as previously unknown and 70 fixed. WhiteFox has been acknowledged by the PyTorch team and is being incorporated into its development workflow. Beyond DL compilers, WhiteFox can also be adapted for compilers in different domains.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 26 of its 86 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

26 of 86 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  10. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 311
  11. 2022Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Modelscited by 103
  12. 2023ChatUniTest: A Framework for LLM-Based Test Generationcited by 96
14 more
  1. 2023Effective Test Generation Using Pre-trained Large Language Models and Mutation Testingcited by 51
  2. 2023Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPTcited by 36
  3. 2018Evaluating Fuzz Testingcited by 32
  4. 2023ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generationcited by 32
  5. 2023Learning Deep Semantics for Test Completioncited by 31
  6. 2022NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilerscited by 30
  7. 2022Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Sourcecited by 28
  8. 2021DocTer: Documentation Guided Fuzzing for Testing Deep Learning API Functionscited by 22
  9. 2023Fuzzing Automatic Differentiation in Deep-Learning Librariescited by 17
  10. 2022Muffin: Testing Deep Learning Libraries via Neural Architecture Fuzzingcited by 16
  11. 2022Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutationcited by 10
  12. 2023NeuRI: Diversifying DNN Generation via Inductive Rule Inferencecited by 7
  13. 2016Skeletal Program Enumeration for Rigorous Compiler Testingcited by 6
  14. 2020Fuzzing with Fast Failure Feedbackcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 17; 2026: 10.

37 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  2. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  3. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 51
  4. 2024AutoVerus: Automated Proof Generation for Rust Codecited by 49
  5. 2025KNighter: Transforming Static Analysis with LLM-Synthesized Checkerscited by 24
  6. 2025ToolFuzz -- Automated Agent Tool Testingcited by 18
  7. 2023KernelGPT: Enhanced Kernel Fuzzing via Large Language Modelscited by 17
  8. 2024LLAMAFUZZ: Large Language Model Enhanced Greybox Fuzzingcited by 14
  9. 2025VeruSAGE: A Study of Agent-Based Verification for Rust Systemscited by 12
  10. 2025ClassInvGen: Class Invariant Synthesis using Large Language Modelscited by 11
  11. 2025Locus: Agentic Predicate Synthesis for Directed Fuzzingcited by 9
  12. 2024When Fuzzing Meets LLMs: Challenges and Opportunitiescited by 8
25 more
  1. 2025Benchmarking LLMs for Unit Test Generation from Real-World Functionscited by 8
  2. 2026LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunitiescited by 8
  3. 2025`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMscited by 7
  4. 2025Validating Network Protocol Parsers with Traceable RFC Document Interpretationcited by 7
  5. 2025Hybrid Fuzzing with LLM-Guided Input Mutation and Semantic Feedbackcited by 7
  6. 2026Test vs Mutant: Adversarial LLM Agents for Robust Unit Test Generationcited by 7
  7. 2024Enhancing Differential Testing With LLMs For Testing Deep Learning Librariescited by 5
  8. 2025Large Language Models for Validating Network Protocol Parserscited by 5
  9. 2024On the Challenges of Fuzzing Techniques via Large Language Modelscited by 4
  10. 2025CTIConnect: A Benchmark for Retrieval-Augmented LLMs over Heterogeneous Cyber Threat Intelligencecited by 4
  11. 2026Agentic Harness for Real-World Compilerscited by 3
  12. 2026FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproductioncited by 3
  13. 2024Generative transformations and patterns in LLM-native approaches for software verification and falsificationcited by 2
  14. 2026SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplacescited by 2
  15. 2025DiTOX: Fault Detection and Localization in the ONNX Optimizercited by 1
  16. 2026LLM-Based Test Case Generation in DBMS through Monte Carlo Tree Searchcited by 1
  17. 2025On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementationscited by 0
  18. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  19. 2025LLMs are All You Need? Improving Fuzz Testing for MOJO with Large Language Modelscited by 0
  20. 2025QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Codecited by 0
  21. 2025Exploring the Feasibility of End-to-End Large Language Model as a Compilercited by 0
  22. 2026The New Compiler Stack: A Survey on the Synergy of LLMs and Compilerscited by 0
  23. 2026Fuzzing with Agents? Generators Are All You Needcited by 0
  24. 2026Understanding Agent-Based Patching of Compiler Missed Optimizationscited by 0
  25. 2026Semantic-aware and Self-improving Program Reduction via Agentic Large Language Modelscited by 0