# Semantic Captioning: Benchmark Dataset and Graph-Aware Few-Shot In-Context Learning for SQL2Text

arXiv: https://arxiv.org/abs/2501.03166 · HTML: https://mlsanity.com/paper/2501.03166/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 6 Jan 2025.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases](https://mlsanity.com/paper/1909.05378/) (2019)
- [SParC: Cross-Domain Semantic Parsing in Context](https://mlsanity.com/paper/1906.02285/) (2019)
- [Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method](https://mlsanity.com/paper/2305.13412/) (2023)
- [In-Context Learning with Iterative Demonstration Selection](https://mlsanity.com/paper/2310.09881/) (2023)
- [A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks](https://mlsanity.com/paper/2010.03648/) (2020)
- [Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning](https://mlsanity.com/paper/2304.11384/) (2023)
- [Dr.ICL: Demonstration-Retrieved In-context Learning](https://mlsanity.com/paper/2305.14128/) (2023)
- [Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners](https://mlsanity.com/paper/2305.14825/) (2023)
- [SummIt: Iterative Text Summarization via ChatGPT](https://mlsanity.com/paper/2305.14835/) (2023)
- [From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting](https://mlsanity.com/paper/2309.04269/) (2023)
- [Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformation](https://mlsanity.com/paper/2310.15515/) (2023)
- [What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?](https://mlsanity.com/paper/2304.07575/) (2023)
- [SQL-to-Text Generation with Graph-to-Sequence Model](https://mlsanity.com/paper/1809.05255/) (2018)
- [GPTutor: a ChatGPT-powered programming tool for code explanation](https://mlsanity.com/paper/2305.01863/) (2023)
- [Injecting the BM25 Score as Text Improves BERT-Based Re-rankers](https://mlsanity.com/paper/2301.09728/) (2023)
- [Exploring Demonstration Ensembling for In-context Learning](https://mlsanity.com/paper/2308.08780/) (2023)
- [SQLfuse: Enhancing Text-to-SQL Performance through Comprehensive LLM Synergy](https://mlsanity.com/paper/2407.14568/) (2024)

## Built on by (most cited first, 1 of 1)

- [SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables](https://mlsanity.com/paper/2602.23286/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable performance in various NLP tasks, including semantic parsing, which translates natural language into formal code representations. However, the reverse process, translating code into natural language, termed semantic captioning, has received less attention. This task is becoming increasingly important as LLMs are integrated into platforms for code generation, security analysis, and educational purposes. In this paper, we focus on the captioning of SQL query (SQL2Text) to address the critical need for understanding and explaining SQL queries in an era where LLM-generated code poses potential security risks. We repurpose Text2SQL datasets for SQL2Text by introducing an iterative ICL prompt using GPT-4o to generate multiple additional utterances, which enhances the robustness of the datasets for the reverse task. We conduct our experiments using in-context learning (ICL) based on different sample selection methods, emphasizing smaller, more computationally efficient LLMs. Our findings demonstrate that leveraging the inherent graph properties of SQL for ICL sample selection significantly outperforms random selection by up to 39% on BLEU score and provides better results than alternative methods. Dataset and codes are published: https://github.com/aliwister/ast-icl.