# GenJoin: Conditional Generative Plan-to-Plan Query Optimizer that Learns from Subplan Hints

arXiv: https://arxiv.org/abs/2411.04525 · HTML: https://mlsanity.com/paper/2411.04525/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 58 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 17 of 17)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Neo: A Learned Query Optimizer](https://mlsanity.com/paper/1904.03711/) (2019)
- [Learned Cardinalities: Estimating Correlated Joins with Deep Learning](https://mlsanity.com/paper/1809.00677/) (2018)
- [Bao: Learning to Steer Query Optimizers](https://mlsanity.com/paper/2004.03814/) (2020)
- [Learning to Optimize Join Queries With Deep Reinforcement Learning](https://mlsanity.com/paper/1808.03196/) (2018)
- [Lero: A Learning-to-Rank Query Optimizer](https://mlsanity.com/paper/2302.06873/) (2023)
- [Deep Reinforcement Learning for Join Order Enumeration](https://mlsanity.com/paper/1803.00055/) (2018)
- [FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation](https://mlsanity.com/paper/2011.09022/) (2020)
- [Balsa: Learning a Query Optimizer Without Expert Demonstrations](https://mlsanity.com/paper/2201.01441/) (2022)
- [Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction](https://mlsanity.com/paper/2201.00561/) (2022)
- [FactorJoin: A New Cardinality Estimation Framework for Join Queries](https://mlsanity.com/paper/2212.05526/) (2022)
- [Towards a Hands-Free Query Optimizer through Deep Learning](https://mlsanity.com/paper/1809.10212/) (2018)
- [Join Query Optimization with Deep Reinforcement Learning Algorithms](https://mlsanity.com/paper/1911.11689/) (2019)
- [COOOL: A Learning-To-Rank Approach for SQL Hint Recommendations](https://mlsanity.com/paper/2304.04407/) (2023)
- [Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective](https://mlsanity.com/paper/2309.01551/) (2023)
- [JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning](https://mlsanity.com/paper/2307.11704/) (2023)

## Built on by (most cited first, 1 of 1)

- [GenDB: The Next Generation of Query Processing -- Synthesized, Not Engineered](https://mlsanity.com/paper/2603.02081/) (2026)

## Abstract (arXiv)

Query optimization has become a research area where classical algorithms are being challenged by machine learning algorithms. At the same time, recent trends in learned query optimizers have shown that it is prudent to take advantage of decades of database research and augment classical query optimizers by shrinking the plan search space through different types of hints (e.g. by specifying the join type, scan type or the order of joins) rather than completely replacing the classical query optimizer with machine learning models. It is especially relevant for cases when classical optimizers cannot fully enumerate all logical and physical plans and, as an alternative, need to rely on less robust approaches like genetic algorithms. However, even symbiotically learned query optimizers are hampered by the need for vast amounts of training data, slow plan generation during inference and unstable results across various workload conditions. In this paper, we present GenJoin - a novel learned query optimizer that considers the query optimization problem as a generative task and is capable of learning from a random set of subplan hints to produce query plans that outperform the classical optimizer. GenJoin is the first learned query optimizer that significantly and consistently outperforms PostgreSQL as well as state-of-the-art methods on two well-known real-world benchmarks across a variety of workloads using rigorous machine learning evaluations.