mlsanityData to 14 Oct 2026

Synthetic Programming Elicitation for Text-to-Code in Very Low-Resource Programming and Formal Languages

arXiv 2406.03636 ↗First posted 5 Jun 2024cs.PL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent advances in large language models (LLMs) for code applications have demonstrated remarkable zero-shot fluency and instruction following on challenging code related tasks ranging from test case generation to self-repair. Unsurprisingly, however, models struggle to compose syntactically valid programs in programming languages unrepresented in pre-training, referred to as very low-resource Programming Languages (VLPLs). VLPLs appear in crucial settings, including domain-specific languages for internal tools, tool-chains for legacy languages, and formal verification frameworks. Inspired by a technique called natural programming elicitation, we propose designing an intermediate language that LLMs "naturally" know how to use and which can be automatically compiled to a target VLPL. When LLMs generate code that lies outside of this intermediate language, we use compiler techniques to repair the code into programs in the intermediate language. Overall, we introduce synthetic programming elicitation and compilation (SPEAC), an approach that enables LLMs to generate syntactically valid code even for VLPLs. We empirically evaluate the performance of SPEAC in a case study for the UCLID5 formal verification language and find that, compared to existing retrieval and fine-tuning baselines, SPEAC produces syntactically correct programs more frequently and without sacrificing semantic correctness.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 22 of its 43 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

22 of 43 references.

References · 22

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  9. 2023Teaching Large Language Models to Self-Debugcited by 988
  10. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  11. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
  12. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
10 more
  1. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
  2. 2024HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalizationcited by 86
  3. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  4. 2024Towards AI-Assisted Synthesis of Verified Dafny Methodscited by 58
  5. 2023Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMscited by 45
  6. 2020Scenic: A Language for Scenario Specification and Data Generationcited by 40
  7. 2022On the Transferability of Pre-trained Language Models for Low-Resource Programming Languagescited by 30
  8. 2023The potential of LLMs for coding with low-resource and domain-specific programming languagescited by 11
  9. 2024ScenicNL: Generating Probabilistic Scenario Programs from Natural Languagecited by 10
  10. 2022UCLID5: Multi-Modal Formal Modeling, Verification, and Synthesiscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 3.

6 papers.

  1. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 66
  2. 2025TreeSynth: Synthesizing Diverse Data from Scratch via Tree-Guided Subspace Partitioningcited by 7
  3. 2026Bridging the Knowledge Void: Inference-time Acquisition of Unfamiliar Programming Languages for Coding Taskscited by 1
  4. 2025Guaranteed Guess: A Language Modeling Approach for CISC-to-RISC Transpilation with Testing Guaranteescited by 0
  5. 2026Learning Context-Free Grammars for Grammar-Constrained Decoding via Declarative Agentic Programming with Guaranteescited by 0
  6. 2026LLM-Based FORM Code Generation with Verification-Driven Fine-Tuningcited by 0