# Learn from Yesterday: A Semi-Supervised Continual Learning Method for Supervision-Limited Text-to-SQL Task Streams

arXiv: https://arxiv.org/abs/2211.11226 · HTML: https://mlsanity.com/paper/2211.11226/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 28 of 28)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation](https://mlsanity.com/paper/1905.08205/) (2019)
- [CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases](https://mlsanity.com/paper/1909.05378/) (2019)
- [GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2009.13845/) (2020)
- [SParC: Cross-Domain Semantic Parsing in Context](https://mlsanity.com/paper/1906.02285/) (2019)
- [LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations](https://mlsanity.com/paper/2106.01093/) (2021)
- [A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization](https://mlsanity.com/paper/1902.01069/) (2019)
- [Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions](https://mlsanity.com/paper/1909.00786/) (2019)
- [Text-to-SQL Generation for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1908.01839/) (2019)
- [Sentence Embedding Alignment for Lifelong Relation Extraction](https://mlsanity.com/paper/1903.02588/) (2019)
- [ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning](https://mlsanity.com/paper/2101.00407/) (2021)
- [Continual Learning for Natural Language Generation in Task-oriented Dialog Systems](https://mlsanity.com/paper/2010.00910/) (2020)
- [Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer](https://mlsanity.com/paper/2101.09536/) (2021)
- [SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising](https://mlsanity.com/paper/2105.07911/) (2021)
- [HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2203.07376/) (2022)
- [StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing](https://mlsanity.com/paper/1806.07832/) (2018)
- [Decoupled Dialogue Modeling and Semantic Parsing for Multi-Turn Text-to-SQL](https://mlsanity.com/paper/2106.02282/) (2021)
- [Hypernetworks for Continual Semi-Supervised Learning](https://mlsanity.com/paper/2110.01856/) (2021)
- [Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers](https://mlsanity.com/paper/2109.05186/) (2021)
- [Learning to Predict Gradients for Semi-Supervised Continual Learning](https://mlsanity.com/paper/2201.09196/) (2022)
- [Leveraging Table Content for Zero-shot Text-to-SQL with Meta-Learning](https://mlsanity.com/paper/2109.05395/) (2021)

## Built on by (most cited first, 7 of 7)

- [Continual Learning for Large Language Models: A Survey](https://mlsanity.com/paper/2402.01364/) (2024)
- [Self-Reinforcing Prototype Evolution with Dual-Knowledge Cooperation for Semi-Supervised Lifelong Person Re-Identification](https://mlsanity.com/paper/2507.01884/) (2025)
- [Parameterizing Context: Unleashing the Power of Parameter-Efficient Fine-Tuning and In-Context Tuning for Continual Table Semantic Parsing](https://mlsanity.com/paper/2310.04801/) (2023)
- [Auto-selected Knowledge Adapters for Lifelong Person Re-identification](https://mlsanity.com/paper/2405.19005/) (2024)
- [K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling](https://mlsanity.com/paper/2509.16929/) (2025)
- [Continual Learning of Domain Knowledge from Human Feedback in Text-to-SQL](https://mlsanity.com/paper/2511.10674/) (2025)
- [Semantic Parsing in Limited Resource Conditions](https://mlsanity.com/paper/2309.07429/) (2023)

## Abstract (arXiv)

Conventional text-to-SQL studies are limited to a single task with a fixed-size training and test set. When confronted with a stream of tasks common in real-world applications, existing methods struggle with the problems of insufficient supervised data and high retraining costs. The former tends to cause overfitting on unseen databases for the new task, while the latter makes a full review of instances from past tasks impractical for the model, resulting in forgetting of learned SQL structures and database schemas. To address the problems, this paper proposes integrating semi-supervised learning (SSL) and continual learning (CL) in a stream of text-to-SQL tasks and offers two promising solutions in turn. The first solution Vanilla is to perform self-training, augmenting the supervised training data with predicted pseudo-labeled instances of the current task, while replacing the full volume retraining with episodic memory replay to balance the training efficiency with the performance of previous tasks. The improved solution SFNet takes advantage of the intrinsic connection between CL and SSL. It uses in-memory past information to help current SSL, while adding high-quality pseudo instances in memory to improve future replay. The experiments on two datasets shows that SFNet outperforms the widely-used SSL-only and CL-only baselines on multiple metrics.