# Neural Program Synthesis with Priority Queue Training

arXiv: https://arxiv.org/abs/1801.03526 · HTML: https://mlsanity.com/paper/1801.03526/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2018.

## Built on by (most cited first, 24 of 24)

- [Self-Imitation Learning](https://mlsanity.com/paper/1806.05635/) (2018)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [Learning to Synthesize Programs as Interpretable and Generalizable Policies](https://mlsanity.com/paper/2108.13643/) (2021)
- [Leveraging Language to Learn Program Abstractions and Search Heuristics](https://mlsanity.com/paper/2106.11053/) (2021)
- [Communicating Natural Programs to Humans and Machines](https://mlsanity.com/paper/2106.07824/) (2021)
- [Deep Generative Symbolic Regression](https://mlsanity.com/paper/2401.00282/) (2023)
- [Generative Adversarial Self-Imitation Learning](https://mlsanity.com/paper/1812.00950/) (2018)
- [Improving exploration in policy gradient search: Application to symbolic optimization](https://mlsanity.com/paper/2107.09158/) (2021)
- [Symbolic Regression via Control Variable Genetic Programming](https://mlsanity.com/paper/2306.08057/) (2023)
- [Hierarchical Neural Program Synthesis](https://mlsanity.com/paper/2303.06018/) (2023)
- [Symbolic Regression for PDEs using Pruned Differentiable Programs](https://mlsanity.com/paper/2303.07009/) (2023)
- [Incorporating domain knowledge into neural-guided search](https://mlsanity.com/paper/2107.09182/) (2021)
- [Vertical Symbolic Regression via Deep Policy Gradient](https://mlsanity.com/paper/2402.00254/) (2024)
- [Racing Control Variable Genetic Programming for Symbolic Regression](https://mlsanity.com/paper/2309.07934/) (2023)
- [BF++: a language for general-purpose program synthesis](https://mlsanity.com/paper/2101.09571/) (2021)
- [Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation](https://mlsanity.com/paper/2211.16838/) (2022)
- [Active Symbolic Discovery of Ordinary Differential Equations via Phase Portrait Sketching](https://mlsanity.com/paper/2409.01416/) (2024)
- [Deep Symbolic Optimization: Reinforcement Learning for Symbolic Mathematics](https://mlsanity.com/paper/2505.10762/) (2025)
- [Neurogenetic Programming Framework for Explainable Reinforcement Learning](https://mlsanity.com/paper/2102.04231/) (2021)
- [EGG-SR: Embedding Symbolic Equivalence into Symbolic Regression via Equality Graph](https://mlsanity.com/paper/2511.05849/) (2025)
- [Vertical Symbolic Regression](https://mlsanity.com/paper/2312.11955/) (2023)
- [Automated Modeling Method for Pathloss Model Discovery](https://mlsanity.com/paper/2505.23383/) (2025)
- [Guided Evolution with Binary Discriminators for ML Program Search](https://mlsanity.com/paper/2402.05821/) (2024)
- [AlgoPilot: Fully Autonomous Program Synthesis Without Human-Written Programs](https://mlsanity.com/paper/2501.06423/) (2025)

## Abstract (arXiv)

We consider the task of program synthesis in the presence of a reward function over the output of programs, where the goal is to find programs with maximal rewards. We employ an iterative optimization scheme, where we train an RNN on a dataset of K best programs from a priority queue of the generated programs so far. Then, we synthesize new programs and add them to the priority queue by sampling from the RNN. We benchmark our algorithm, called priority queue training (or PQT), against genetic algorithm and reinforcement learning baselines on a simple but expressive Turing complete programming language called BF. Our experimental results show that our simple PQT algorithm significantly outperforms the baselines. By adding a program length penalty to the reward function, we are able to synthesize short, human readable programs.