# EvoSchema: Towards Text-to-SQL Robustness Against Schema Evolution

arXiv: https://arxiv.org/abs/2603.10697 · HTML: https://mlsanity.com/paper/2603.10697/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel](https://mlsanity.com/paper/2304.11277/) (2023)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [CodeS: Towards Building Open-source Language Models for Text-to-SQL](https://mlsanity.com/paper/2402.16347/) (2024)
- [Structure-Grounded Pretraining for Text-to-SQL](https://mlsanity.com/paper/2010.12773/) (2020)
- [Exploring Chain-of-Thought Style Prompting for Text-to-SQL](https://mlsanity.com/paper/2305.14215/) (2023)
- [ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought](https://mlsanity.com/paper/2310.17342/) (2023)
- [Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness](https://mlsanity.com/paper/2301.08881/) (2023)
- [Robust Text-to-SQL Generation with Execution-Guided Decoding](https://mlsanity.com/paper/1807.03100/) (2018)
- [FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysis](https://mlsanity.com/paper/2401.10506/) (2024)
- [SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Models](https://mlsanity.com/paper/2403.02951/) (2024)
- [Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation](https://mlsanity.com/paper/2212.09994/) (2022)
- [Enhancing Text-to-SQL Translation for Financial System Design](https://mlsanity.com/paper/2312.14725/) (2023)
- [Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queries](https://mlsanity.com/paper/2402.08349/) (2024)
- [Towards Generalizable and Robust Text-to-SQL Parsing](https://mlsanity.com/paper/2210.12674/) (2022)
- [Federated Learning for Semantic Parsing: Task Formulation, Evaluation Setup, New Algorithms](https://mlsanity.com/paper/2305.17221/) (2023)

## Built on by (most cited first, 5 of 5)

- [TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas](https://mlsanity.com/paper/2603.16448/) (2026)
- [TACO: Task-Aware Column Description Generation Using LLMs](https://mlsanity.com/paper/2606.21685/) (2026)
- [Benchmarking Text-to-SQL under Role-Based Access Control](https://mlsanity.com/paper/2607.22115/) (2026)
- [ISEE: Interactive Semantic Enrichment for Database Fields](https://mlsanity.com/paper/2608.02604/) (2026)
- [TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysis](https://mlsanity.com/paper/2608.24145/) (2026)

## Abstract (arXiv)

Neural text-to-SQL models, which translate natural language questions (NLQs) into SQL queries given a database schema, have achieved remarkable performance. However, database schemas frequently evolve to meet new requirements. Such schema evolution often leads to performance degradation for models trained on static schemas. Existing work either mainly focuses on simply paraphrasing some syntactic or semantic mappings among NLQ, DB and SQL, or lacks a comprehensive and controllable way to investigate the model robustness issue under the schema evolution, which is insufficient when facing the increasingly complex and rich database schema changes in reality, especially in the LLM era. To address the challenges posed by schema evolution, we present EvoSchema, a comprehensive benchmark designed to assess and enhance the robustness of text-to-SQL systems under real-world schema changes. EvoSchema introduces a novel schema evolution taxonomy, encompassing ten perturbation types across columnlevel and table-level modifications, systematically simulating the dynamic nature of database schemas. Through EvoSchema, we conduct an in-depth evaluation spanning different open source and closed-source LLMs, revealing that table-level perturbations have a significantly greater impact on model performance compared to column-level changes. Furthermore, EvoSchema inspires the development of more resilient text-to-SQL systems, in terms of both model training and database design. The models trained on EvoSchema's diverse schema designs can force the model to distinguish the schema difference for the same questions to avoid learning spurious patterns, which demonstrate remarkable robustness compared to those trained on unperturbed data on average. This benchmark offers valuable insights into model behavior and a path forward for designing systems capable of thriving in dynamic, real-world environments.