# A Static Analyzer for Detecting Tensor Shape Errors in Deep Neural Network Training Code

arXiv: https://arxiv.org/abs/2112.09037 · HTML: https://mlsanity.com/paper/2112.09037/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 16 Dec 2021.

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Ariadne: Analysis for Machine Learning Program](https://mlsanity.com/paper/1805.04058/) (2018)
- [ShapeFlow: Dynamic Shape Interpreter for TensorFlow](https://mlsanity.com/paper/2011.13452/) (2020)

## Built on by (most cited first, 2 of 2)

- [Training with Confidence: Catching Silent Errors in Deep Learning Training with Automated Proactive Checks](https://mlsanity.com/paper/2506.14813/) (2025)
- [NeuRI: Diversifying DNN Generation via Inductive Rule Inference](https://mlsanity.com/paper/2302.02261/) (2023)

## Abstract (arXiv)

We present an automatic static analyzer PyTea that detects tensor-shape errors in PyTorch code. The tensor-shape error is critical in the deep neural net code; much of the training cost and intermediate results are to be lost once a tensor shape mismatch occurs in the midst of the training phase. Given the input PyTorch source, PyTea statically traces every possible execution path, collects tensor shape constraints required by the tensor operation sequence of the path, and decides if the constraints are unsatisfiable (hence a shape error can occur). PyTea's scalability and precision hinges on the characteristics of real-world PyTorch applications: the number of execution paths after PyTea's conservative pruning rarely explodes and loops are simple enough to be circumscribed by our symbolic abstraction. We tested PyTea against the projects in the official PyTorch repository and some tensor-error code questioned in the StackOverflow. PyTea successfully detects tensor shape errors in these codes, each within a few seconds.