mlsanityData to 14 Oct 2026

Context-Aware Distillation and Ablation for Text2DSL

arXiv 2606.22578 ↗First posted 21 Jun 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We extend our prior work on Text2DSL automatic generation of domain-specific language (DSL) code from natural language descriptions along two complementary axes. First, we replace prompt-only synthetic generation with context-aware distillation, in which a teacher large language model (DeepSeek-V4-Flash) operates under an explicitly defined structured context comprising a BNF grammar, an API specification, and a closed identifier vocabulary; the resulting corpus is verified by a two-tier pipeline combining AST validation through esprima and runtime acceptance through the production polkitd daemon and the pkcheck client. This scales the verified PolkitBench corpus from 4,204 to 10,073 natural-language-to-Polkit-rule pairs at 100.0% AST validity and 99.7% runtime pass rate. Second, we conduct the per-component factorial ablation of structured context that was identified as future work in the precursor study: eight conditions C0-C7 are evaluated on GigaChat-10B-A1.8B with the new corpus. Three findings emerge. (i) The new harder corpus collapses the baseline mode (Syntax Valid 97.6% -> 58.5%, Combined Score 0.482 -> 0.252), whereas the context-enhanced mode degrades only marginally (Syntax 98.6% -> 97.4%, Combined 0.801 -> 0.750), confirming that structured context is not a cosmetic improvement but a load-bearing mechanism. (ii) The best absolute condition is the full context C7 across all metrics, while the strongest partial conditions (C5 = BNF + Vocabulary, C6 = API + Vocabulary) both contain the vocabulary. (iii) A Shapley-style decomposition assigns the largest semantic-quality effect to the vocabulary (Combined +0.198), the largest structural-validity effects to API (+24.7 pp) and BNF (+22.3 pp).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 21 Jun 2026.

Builds on

21 of 32 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  9. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  10. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  11. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  12. 2023StarCoder: may the source be with you!cited by 799
9 more
  1. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  2. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  3. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  4. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  5. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  6. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 195
  7. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  8. 2021Jigsaw: Large Language Models meet Program Synthesiscited by 103
  9. 2022GLITCH: Automated Polyglot Security Smell Detection in Infrastructure as Codecited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Text2DSL: LLM-Based Code Generation for Domain-Specific Languages
  2. 2026Context-Instrumental Data Distillation for Kubernetes Manifest Generation: Method and Experimental Evaluation
  3. 2026Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data
  4. 2025Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
  5. 2024Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity
  6. 2026A Factorial Study of Synthetic Data Generation for Low-Resource Machine Translation using Grammar Books
  7. 2026DS2-Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning
  8. 2025Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation