Neo: A Learned Query Optimizer
Not on the map. Why?
Abstract (arXiv)
Query optimization is one of the most challenging problems in database systems. Despite the progress made over the past decades, query optimizers remain extremely complex components that require a great deal of hand-tuning for specific workloads and datasets. Motivated by this shortcoming and inspired by recent advances in applying machine learning to data management challenges, we introduce Neo (Neural Optimizer), a novel learning-based query optimizer that relies on deep neural networks to generate query executions plans. Neo bootstraps its query optimization model from existing optimizers and continues to learn from incoming queries, building upon its successes and learning from its failures. Furthermore, Neo naturally adapts to underlying data patterns and is robust to estimation errors. Experimental results demonstrate that Neo, even when bootstrapped from a simple optimizer like PostgreSQL, can learn a model that offers similar performance to state-of-the-art commercial optimizers, and in some cases even surpass them.
Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 15; 2022: 11; 2023: 7; 2024: 12; 2025: 22; 2026: 13.
80 papers.
Sort:
By area:
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
- 2025Barbarians at the Gate: How AI is Upending Systems Researchcited by 48
- 2023Lero: A Learning-to-Rank Query Optimizercited by 26
- 2021DB-BERT: a Database Tuning Tool that "Reads the Manual"cited by 19
- 2022Balsa: Learning a Query Optimizer Without Expert Demonstrationscited by 18
- 2022Zero-Shot Cost Models for Out-of-the-box Learned Cost Predictioncited by 16
- 2021A Survey on Advancing the DBMS Query Optimizer: Cardinality Estimation, Cost Model, and Plan Enumerationcited by 14
- 2025Glia: A Human-Inspired AI for Automated Systems Design and Optimizationcited by 14
- 2022FactorJoin: A New Cardinality Estimation Framework for Join Queriescited by 13
- 2025Cortex AISQL: A Production SQL Engine for Unstructured Datacited by 11
- 2021A Unified Transferable Model for ML-Enhanced DBMScited by 10
- 2021One Model to Rule them All: Towards Zero-Shot Learning for Databasescited by 9
38 more
- 2021Toward Code Generation: A Survey and Lessons from Semantic Parsingcited by 8
- 2022Detect, Distill and Update: Learned DB Systems Facing Out of Distribution Datacited by 6
- 2023Online Reinforcement Learning in Non-Stationary Context-Driven Environmentscited by 6
- 2025Graph Neural Networks for Databases: A Surveycited by 6
- 2025QUITE: A Query Rewrite System Beyond Rules with LLM Agentscited by 6
- 2025Batch Query Processing and Optimization for Agentic Workflowscited by 6
- 2022CodexDB: Generating Code for Processing SQL Queries using GPT-3 Codexcited by 5
- 2024NeurDB: An AI-powered Autonomous Data Systemcited by 5
- 2021Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workloadcited by 4
- 2021Learned Sorted Table Search and Static Indexes in Small Model Spacecited by 4
- 2022Learning to Improve Code Efficiencycited by 4
- 2023Real-time Workload Pattern Analysis for Large-scale Cloud Databasescited by 4
- 2024Sibyl: Forecasting Time-Evolving Query Workloadscited by 4
- 2024A Survey of Learned Indexes for the Multi-dimensional Spacecited by 4
- 2025RCRank: Multimodal Ranking of Root Causes of Slow Queries in Cloud Database Systemscited by 4
- 2023Kepler: Robust Learning for Faster Parametric Query Optimizationcited by 3
- 2024AnnotatedTables: A Large Tabular Dataset with Language Model Annotationscited by 3
- 2024Learned Graph Rewriting with Equality Saturation: A New Paradigm in Relational Query Rewrite and Beyondcited by 3
- 2024HERO: Hint-Based Efficient and Reliable Query Optimizercited by 3
- 2021Can Deep Neural Networks Predict Data Correlations from Column Names?cited by 2
- 2022Demystifying Reinforcement Learning in Time-Varying Systemscited by 2
- 2022Deploying a Steered Query Optimizer in Production at Microsoftcited by 2
- 2022Learned-Database Systems Securitycited by 2
- 2023ML-Powered Index Tuning: An Overview of Recent Progress and Open Challengescited by 2
- 2025Towards Agentic OS: An LLM Agent Framework for Linux Schedulerscited by 2
- 2025Supporting Dynamic Agentic Workloads: How Data and Agents Interactcited by 2
- 2025Let the Barbarians In: How AI Can Accelerate Systems Performance Researchcited by 2
- 2025Cost Trade-offs of Reasoning and Non-Reasoning Large Language Models in Text-to-SQLcited by 2
- 2026AI-Driven Research for Databasescited by 2
- 2021A Pluggable Learned Index Method via Sampling and Gap Insertioncited by 1
- 2021TENSILE: A Tensor granularity dynamic GPU memory scheduling method toward multiple dynamic workloads systemcited by 1
- 2021Phoebe: A Learning-based Checkpoint Optimizercited by 1
- 2023JoinGym: An Efficient Query Optimization Environment for Reinforcement Learningcited by 1
- 2023FOSS: A Self-Learned Doctor for Query Optimizercited by 1
- 2024GEqO: ML-Accelerated Semantic Equivalence Detectioncited by 1
- 2024Robust Plan Evaluation based on Approximate Probabilistic Machine Learningcited by 1
- 2024A Novel Technique for Query Plan Representation Based on Graph Neural Netscited by 1
- 2024Hype or Heuristic? Quantum Reinforcement Learning for Join Order Optimisationcited by 1
50 most cited of 80 All 80 (JSON)
- 2026Towards Anticipatory Databases Through Shared Data and Workload Semantics13 Sep 2026
- 2026VikingRAG: Accurate and Token-efficient Retrieval-augmented Generation over Structured Documents10 Sep 2026
- 2026DataKernelBench: Can LLMs Optimize Database Queries on GPUs?25 Aug 2026
- 2026Fully Inductive Cardinality Estimation30 Jul 2026
- 2026Selectivity Estimation for Linear Queries via Online Learning3 Jul 2026
- 2026The Time is Here for Just-in-Time Systems: Challenges and Opportunities22 May 2026
- 2026Agentic Cost-Aware Query Planning with Knowledge Distillation for Big Data Analytics18 May 2026
- 2026Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization28 Apr 2026
- 2026Pre-Execution Query Slot-Time Prediction in Cloud Data Warehouses: A Feature-Scoped Machine Learning Approach22 Apr 2026
- 2026RELOAD: A Robust and Efficient Learned Query Optimizer for Database Systems16 Apr 2026
- 2026AI-Driven Research for Databases8 Apr 2026
- 2026MICRO: A Lightweight Middleware for Optimizing Cross-store Cross-model Graph-Relation Joins [Technical Report]14 Mar 2026
38 more
- 2026Piece of CAKE: Adaptive Execution Engines via Microsecond-Scale Learning4 Feb 2026
- 2025Cost Trade-offs of Reasoning and Non-Reasoning Large Language Models in Text-to-SQL26 Dec 2025
- 2025Let the Barbarians In: How AI Can Accelerate Systems Performance Research16 Dec 2025
- 2025Supporting Dynamic Agentic Workloads: How Data and Agents Interact10 Dec 2025
- 2025QJoin: Transformation-aware Joinable Data Discovery Using Reinforcement Learning2 Dec 2025
- 2025Gradient-Based Join Ordering18 Nov 2025
- 2025Cortex AISQL: A Production SQL Engine for Unstructured Data10 Nov 2025
- 2025Glia: A Human-Inspired AI for Automated Systems Design and Optimization31 Oct 2025
- 2025GrASP: A Generalizable Address-based Semantic Prefetcher for Scalable Transactional and Analytical Workloads13 Oct 2025
- 2025Barbarians at the Gate: How AI is Upending Systems Research7 Oct 2025
- 2025Batch Query Processing and Optimization for Agentic Workflows2 Sep 2025
- 2025Towards Agentic OS: An LLM Agent Framework for Linux Schedulers1 Sep 2025
- 2025Bootstrapping Learned Cost Models with Synthetic SQL Queries27 Aug 2025
- 2025AutoIndexer: A Reinforcement Learning-Enhanced Index Advisor Towards Scaling Workloads30 Jul 2025
- 2025QUITE: A Query Rewrite System Beyond Rules with LLM Agents9 Jun 2025
- 2025Training-Free Query Optimization via LLM-Based Plan Similarity6 Jun 2025
- 2025Low Rank Learning for Offline Query Optimization8 Apr 2025
- 2025DistJoin: A Decoupled Join Cardinality Estimator based on Adaptive Neural Predicate Modulation12 Mar 2025
- 2025Large Scale Multi-Task Bayesian Optimization with Large Language Models11 Mar 2025
- 2025RCRank: Multimodal Ranking of Root Causes of Slow Queries in Cloud Database Systems6 Mar 2025
- 2025Selective Use of Yannakakis' Algorithm to Improve Query Performance: Machine Learning to the Rescue27 Feb 2025
- 2025Graph Neural Networks for Databases: A Survey18 Feb 2025
- 2025Reqo: A Comprehensive Learning-Based Cost Model for Robust and Explainable Query Optimization29 Jan 2025
- 2024HERO: Hint-Based Efficient and Reliable Query Optimizer3 Dec 2024
- 2024CONCERTO: Complex Query Execution Mechanism-Aware Learned Cost Estimation1 Dec 2024
- 2024GenJoin: Conditional Generative Plan-to-Plan Query Optimizer that Learns from Subplan Hints7 Nov 2024
- 2024AnnotatedTables: A Large Tabular Dataset with Language Model Annotations24 Jun 2024
- 2024Learned Graph Rewriting with Equality Saturation: A New Paradigm in Relational Query Rewrite and Beyond19 Jun 2024
- 2024Hype or Heuristic? Quantum Reinforcement Learning for Join Order Optimisation13 May 2024
- 2024A Novel Technique for Query Plan Representation Based on Graph Neural Nets8 May 2024
- 2024NeurDB: An AI-powered Autonomous Data System7 May 2024
- 2024A Survey of Learned Indexes for the Multi-dimensional Space11 Mar 2024
- 2024Robust Plan Evaluation based on Approximate Probabilistic Machine Learning26 Jan 2024
- 2024Sibyl: Forecasting Time-Evolving Query Workloads8 Jan 2024
- 2024GEqO: ML-Accelerated Semantic Equivalence Detection2 Jan 2024
- 2023FOSS: A Self-Learned Doctor for Query Optimizer11 Dec 2023
- 2023ML-Powered Index Tuning: An Overview of Recent Progress and Open Challenges25 Aug 2023
- 2023JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning21 Jul 2023
50 newest of 80 All 80 (JSON)
- 2025QUITE: A Query Rewrite System Beyond Rules with LLM Agentscited by 6
- 2025Towards Agentic OS: An LLM Agent Framework for Linux Schedulerscited by 2
- 2026Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimizationcited by 1
- 2026Agentic Cost-Aware Query Planning with Knowledge Distillation for Big Data Analyticscited by 0
- 2026Agentic Cost-Aware Query Planning with Knowledge Distillation for Big Data Analytics18 May 2026
- 2026Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization28 Apr 2026
- 2025Towards Agentic OS: An LLM Agent Framework for Linux Schedulers1 Sep 2025
- 2025QUITE: A Query Rewrite System Beyond Rules with LLM Agents9 Jun 2025
- 2025Bootstrapping Learned Cost Models with Synthetic SQL Queries27 Aug 2025
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
- 2025Barbarians at the Gate: How AI is Upending Systems Researchcited by 48
- 2023Lero: A Learning-to-Rank Query Optimizercited by 26
- 2021DB-BERT: a Database Tuning Tool that "Reads the Manual"cited by 19
- 2022Balsa: Learning a Query Optimizer Without Expert Demonstrationscited by 18
- 2022Zero-Shot Cost Models for Out-of-the-box Learned Cost Predictioncited by 16
- 2021A Survey on Advancing the DBMS Query Optimizer: Cardinality Estimation, Cost Model, and Plan Enumerationcited by 14
- 2025Glia: A Human-Inspired AI for Automated Systems Design and Optimizationcited by 14
- 2022FactorJoin: A New Cardinality Estimation Framework for Join Queriescited by 13
- 2025Cortex AISQL: A Production SQL Engine for Unstructured Datacited by 11
- 2021A Unified Transferable Model for ML-Enhanced DBMScited by 10
- 2021One Model to Rule them All: Towards Zero-Shot Learning for Databasescited by 9
12 most cited of 71 All 71 (JSON)
- 2026Towards Anticipatory Databases Through Shared Data and Workload Semantics13 Sep 2026
- 2026DataKernelBench: Can LLMs Optimize Database Queries on GPUs?25 Aug 2026
- 2026Fully Inductive Cardinality Estimation30 Jul 2026
- 2026Selectivity Estimation for Linear Queries via Online Learning3 Jul 2026
- 2026The Time is Here for Just-in-Time Systems: Challenges and Opportunities22 May 2026
- 2026Pre-Execution Query Slot-Time Prediction in Cloud Data Warehouses: A Feature-Scoped Machine Learning Approach22 Apr 2026
- 2026RELOAD: A Robust and Efficient Learned Query Optimizer for Database Systems16 Apr 2026
- 2026AI-Driven Research for Databases8 Apr 2026
- 2026MICRO: A Lightweight Middleware for Optimizing Cross-store Cross-model Graph-Relation Joins [Technical Report]14 Mar 2026
- 2026Piece of CAKE: Adaptive Execution Engines via Microsecond-Scale Learning4 Feb 2026
- 2025Let the Barbarians In: How AI Can Accelerate Systems Performance Research16 Dec 2025
- 2025QJoin: Transformation-aware Joinable Data Discovery Using Reinforcement Learning2 Dec 2025
12 newest of 71 All 71 (JSON)