mlsanityData to 14 Oct 2026

Do LLMs Align with My Task? Evaluating Text-to-SQL via Dataset Alignment

arXiv 2510.04919 ↗First posted 6 Oct 2025cs.CL cs.AI cs.DBcited by 0

Not on the map. Why?

Abstract (arXiv)

Supervised Fine-Tuning (SFT) is an effective method for adapting Large Language Models (LLMs) on downstream tasks. However, variability in training data can hinder a model's ability to generalize across domains. This paper studies the problem of dataset alignment for Natural Language to SQL (NL2SQL or text to SQL), examining how well SFT training data matches the structural characteristics of target queries and how this alignment impacts model performance. We hypothesize that alignment can be accurately estimated by comparing the distributions of structural SQL features across the training set, target data, and the model's predictions prior to SFT. Through comprehensive experiments on three large cross-domain NL2SQL benchmarks and multiple model families, we show that structural alignment is a strong predictor of fine-tuning success. When alignment is high, SFT yields substantial gains in accuracy and SQL generation quality; when alignment is low, improvements are marginal or absent. These findings highlight the importance of alignment-aware data selection for effective fine-tuning and generalization in NL2SQL tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

24 of 29 references.

References · 24

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  4. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  7. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  8. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  9. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  10. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  11. 2023Data Selection for Language Models via Importance Resamplingcited by 329
  12. 2024A Survey on Data Selection for Language Modelscited by 236
12 more
  1. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  2. 2024BOND: Aligning LLMs with Best-of-N Distillationcited by 76
  3. 2024Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Modelscited by 65
  4. 2024Synthesizing Text-to-SQL Data from Weak and Strong LLMscited by 54
  5. 2024DTS-SQL: Decomposed Text-to-SQL with Small Large Language Modelscited by 45
  6. 2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
  7. 2023MFTCoder: Boosting Code LLMs with Multitask Fine-Tuningcited by 35
  8. 2024Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMscited by 32
  9. 2024Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruningcited by 20
  10. 2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Mergingcited by 16
  11. 2017Question Answering through Transfer Learning from Large Fine-grained Supervision Datacited by 9
  12. 2024SQL-Encoder: Improving NL2SQL In-Context Learning Through a Context-Aware Encodercited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL
  2. 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
  3. 2025Quantifying the Importance of Data Alignment in Downstream Model Performance
  4. 2024Efficient Alignment of Large Language Models via Data Sampling
  5. 2025Data Selection for LLM Alignment Using Fine-Grained Preferences
  6. 2026Optimizing Small Language Models for NL2SQL via Chain-of-Thought Fine-Tuning
  7. 2026From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
  8. 2025A Comprehensive Evaluation framework of Alignment Techniques for LLMs