# Graph-Based Fuzz Testing for Deep Learning Inference Engine

arXiv: https://arxiv.org/abs/2008.05933 · HTML: https://mlsanity.com/paper/2008.05933/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 7 of 7)

- [NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers](https://mlsanity.com/paper/2207.13066/) (2022)
- [NeuRI: Diversifying DNN Generation via Inductive Rule Inference](https://mlsanity.com/paper/2302.02261/) (2023)
- [Enhancing Differential Testing With LLMs For Testing Deep Learning Libraries](https://mlsanity.com/paper/2406.07944/) (2024)
- [COMET: Coverage-guided Model Generation For Deep Learning Library Testing](https://mlsanity.com/paper/2208.01508/) (2022)
- [Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing](https://mlsanity.com/paper/2605.11202/) (2026)
- [ConFL: Constraint-guided Fuzzing for Machine Learning Framework](https://mlsanity.com/paper/2307.05642/) (2023)
- [Verify Distributed Deep Learning Model Implementation Refinement with Iterative Relation Inference](https://mlsanity.com/paper/2508.09505/) (2025)

## Abstract (arXiv)

With the wide use of Deep Learning (DL) systems, academy and industry begin to pay attention to their quality. Testing is one of the major methods of quality assurance. However, existing testing techniques focus on the quality of DL models but lacks attention to the core underlying inference engines (i.e., frameworks and libraries). Inspired by the success stories of fuzz testing, we design a graph-based fuzz testing method to improve the quality of DL inference engines. This method is naturally followed by the graph structure of DL models. A novel operator-level coverage criterion based on graph theory is introduced and six different mutations are implemented to generate diversified DL models by exploring combinations of model structures, parameters, and data inputs. The Monte Carlo Tree Search (MCTS) is used to drive DL model generation without a training process. The experimental results show that the MCTS outperforms the random method in boosting operator-level coverage and detecting exceptions. Our method has discovered more than 40 different exceptions in three types of undesired behaviors: model conversion failure, inference failure, output comparison failure. The mutation strategies are useful to generate new valid test inputs, by up to 8.2% more operator-level coverage on average and 8.6 more exceptions captured.