# ast2vec: Utilizing Recursive Neural Encodings of Python Programs

arXiv: https://arxiv.org/abs/2103.11614 · HTML: https://mlsanity.com/paper/2103.11614/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 67 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Syntax-Directed Variational Autoencoder for Structured Data](https://mlsanity.com/paper/1802.08786/) (2018)
- [Tree-to-tree Neural Networks for Program Translation](https://mlsanity.com/paper/1802.03691/) (2018)
- [Learning Linear Dynamical Systems via Spectral Filtering](https://mlsanity.com/paper/1711.00946/) (2017)
- [Automated Clustering and Program Repair for Introductory Programming Assignments](https://mlsanity.com/paper/1603.03165/) (2016)
- [Learning Program Embeddings to Propagate Feedback on Student Code](https://mlsanity.com/paper/1505.05969/) (2015)
- [The Continuous Hint Factory - Providing Hints in Vast and Sparsely Populated Edit Distance Spaces](https://mlsanity.com/paper/1708.06564/) (2017)
- [A Survey of Automated Programming Hint Generation -- The HINTS Framework](https://mlsanity.com/paper/1908.11566/) (2019)
- [Time-varying Learning and Content Analytics via Sparse Factor Analysis](https://mlsanity.com/paper/1312.5734/) (2013)
- [Metric and non-metric proximity transformations at linear costs](https://mlsanity.com/paper/1411.1646/) (2014)
- [Recursive Tree Grammar Autoencoders](https://mlsanity.com/paper/2012.02097/) (2020)
- [Latent Skill Embedding for Personalized Lesson Sequence Recommendation](https://mlsanity.com/paper/1602.07029/) (2016)
- [Time Series Prediction for Graphs in Kernel and Dissimilarity Spaces](https://mlsanity.com/paper/1704.06498/) (2017)

## Abstract (arXiv)

Educational datamining involves the application of datamining techniques to student activity. However, in the context of computer programming, many datamining techniques can not be applied because they expect vector-shaped input whereas computer programs have the form of syntax trees. In this paper, we present ast2vec, a neural network that maps Python syntax trees to vectors and back, thereby facilitating datamining on computer programs as well as the interpretation of datamining results. Ast2vec has been trained on almost half a million programs of novice programmers and is designed to be applied across learning tasks without re-training, meaning that users can apply it without any need for (additional) deep learning. We demonstrate the generality of ast2vec in three settings: First, we provide example analyses using ast2vec on a classroom-sized dataset, involving visualization, student motion analysis, clustering, and outlier detection, including two novel analyses, namely a progress-variance-projection and a dynamical systems analysis. Second, we consider the ability of ast2vec to recover the original syntax tree from its vector representation on the training data and two further large-scale programming datasets. Finally, we evaluate the predictive capability of a simple linear regression on top of ast2vec, obtaining similar results to techniques that work directly on syntax trees. We hope ast2vec can augment the educational datamining toolbelt by making analyses of computer programs easier, richer, and more efficient.