# Autoencoders as Tools for Program Synthesis

arXiv: https://arxiv.org/abs/2108.07129 · HTML: https://mlsanity.com/paper/2108.07129/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 16 Aug 2021.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing](https://mlsanity.com/paper/1903.10145/) (2019)
- [Tree-to-tree Neural Networks for Program Translation](https://mlsanity.com/paper/1802.03691/) (2018)
- [Abstract Syntax Networks for Code Generation and Semantic Parsing](https://mlsanity.com/paper/1704.07535/) (2017)
- [Efficient softmax approximation for GPUs](https://mlsanity.com/paper/1609.04309/) (2016)
- [Variational Recurrent Auto-Encoders](https://mlsanity.com/paper/1412.6581/) (2014)
- [On Learning Meaningful Code Changes via Neural Machine Translation](https://mlsanity.com/paper/1901.09102/) (2019)
- [Code Completion with Neural Attention and Pointer Networks](https://mlsanity.com/paper/1711.09573/) (2017)
- [Program Synthesis and Semantic Parsing with Learned Code Idioms](https://mlsanity.com/paper/1906.10816/) (2019)
- [Variational Autoencoder with Implicit Optimal Priors](https://mlsanity.com/paper/1809.05284/) (2018)
- [Aroma: Code Recommendation via Structural Code Search](https://mlsanity.com/paper/1812.01158/) (2018)
- [A Hierarchical Neural Autoencoder for Paragraphs and Documents](https://mlsanity.com/paper/1506.01057/) (2015)
- [Automatic Software Repair: a Bibliography](https://mlsanity.com/paper/1807.00515/) (2018)
- [Attention-Based LSTM for Psychological Stress Detection from Spoken Language Using Distant Supervision](https://mlsanity.com/paper/1805.12307/) (2018)
- [On Posterior Collapse and Encoder Feature Dispersion in Sequence VAEs](https://mlsanity.com/paper/1911.03976/) (2019)
- [Structural Language Models of Code](https://mlsanity.com/paper/1910.00577/) (2019)
- [A Comparison of Pooling Methods on LSTM Models for Rare Acoustic Event Classification](https://mlsanity.com/paper/2002.06279/) (2020)
- [Neurogenetic Programming Framework for Explainable Reinforcement Learning](https://mlsanity.com/paper/2102.04231/) (2021)
- [Learning to superoptimize programs - Workshop Version](https://mlsanity.com/paper/1612.01094/) (2016)

## Built on by (most cited first, 4 of 4)

- [Fully Autonomous Programming with Large Language Models](https://mlsanity.com/paper/2304.10423/) (2023)
- [Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models](https://mlsanity.com/paper/2503.07693/) (2025)
- [BenchPress: A Deep Active Benchmark Generator](https://mlsanity.com/paper/2208.06555/) (2022)
- [BenchDirect: A Directed Language Model for Compiler Benchmarks](https://mlsanity.com/paper/2303.01557/) (2023)

## Abstract (arXiv)

Recently there have been many advances in research on language modeling of source code. Applications range from code suggestion and completion to code summarization. However, complete program synthesis of industry-grade programming languages remains an open problem. In this work, we introduce and experimentally validate a variational autoencoder model for program synthesis of industry-grade programming languages. This model makes use of the inherent tree structure of code and can be used in conjunction with gradient free optimization techniques like evolutionary methods to generate programs that maximize a given fitness function, for instance, passing a set of test cases. A demonstration is avaliable at https://tree2tree.app