mlsanityData to 14 Oct 2026

Generating Data for Symbolic Language with Large Language Models

arXiv 2305.13917 ↗First posted 23 May 2023cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

While large language models (LLMs) bring not only performance but also complexity, recent work has started to turn LLMs into data generators rather than task inferencers, where another affordable task model is trained for efficient deployment and inference. However, such an approach has primarily been applied to natural language tasks and has not yet been explored for symbolic language tasks with complex structured outputs (e.g., semantic parsing and code generation). In this paper, we propose SymGen which utilizes LLMs for generating various annotation-expensive symbolic language data. SymGen consists of an informative prompt to steer generation and an agreement-based verifier to improve data correctness. We conduct extensive experiments on six symbolic language tasks across various settings. Compared with the LLMs, we demonstrate the 1%-sized task model can achieve comparable or better performance, largely cutting inference and deployment costs. We also show that generated data with only a few human demonstrations can be as effective as over 10 times the amount of human-annotated data when training the task model, saving a considerable amount of annotation effort. SymGen sheds new light on data generation for complex tasks, and we release the code at hrefhttps://github.com/HKUNLP/SymGenhttps://github.com/HKUNLP/SymGen.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 59 of its 88 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

59 of 88 references.

References · 59

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  12. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
38 more
  1. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  2. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  3. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  4. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  5. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  6. 2022Transformers learn in-context by gradient descentcited by 732
  7. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  8. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  9. 2022CodeT: Code Generation with Generated Testscited by 384
  10. 2021Random Feature Attentioncited by 354
  11. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  12. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  13. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  14. 2021Generating Datasets with Pretrained Language Modelscited by 193
  15. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  16. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
  17. 2022Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Orderingcited by 161
  18. 2019Good-Enough Compositional Data Augmentationcited by 158
  19. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 158
  20. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  21. 2023Compositional Exemplars for In-context Learningcited by 144
  22. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  23. 2020Semantic Evaluation for Text-to-SQL with Distilled Test Suitescited by 126
  24. 2022Natural Language to Code Translation with Executioncited by 124
  25. 2016Data Recombination for Neural Semantic Parsingcited by 111
  26. 2018Semantic Parsing for Task Oriented Dialog using Hierarchical Representationscited by 101
  27. 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
  28. 2017Learning a Neural Semantic Parser from User Feedbackcited by 99
  29. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 98
  30. 2022Language Models Can Teach Themselves to Program Bettercited by 78
  31. 2020Grounded Adaptation for Zero-shot Executable Semantic Parsingcited by 76
  32. 2018SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Taskcited by 74
  33. 2020Sequence-Level Mixed Sample Data Augmentationcited by 71
  34. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
  35. 2020Learning to Recombine and Resample Data for Compositional Generalizationcited by 65
  36. 2021Few-Shot Semantic Parsing with Language Models Trained On Codecited by 50
  37. 2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50
  38. 2021Improving Compositional Generalization with Latent Structure and Data Augmentationcited by 46

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 5; 2026: 2.

12 papers.

  1. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  2. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
  3. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  4. 2023FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answeringcited by 51
  5. 2025IC-Cache: Efficient Large Language Model Serving via In-context Cachingcited by 13
  6. 2024XMainframe: A Large Language Model for Mainframe Modernizationcited by 6
  7. 2024Fine-grainedly Synthesize Streaming Data Based On Large Language Models With Graph Structure Understanding For Data Sparsitycited by 1
  8. 2025SoK: Data Minimization in Machine Learningcited by 1
  9. 2025NL2GenSym: Natural Language to Generative Symbolic Rules for SOAR Cognitive Architecture via Large Language Modelscited by 1
  10. 2026Adversarial Arena: Crowdsourcing Data Generation through Interactive Competitioncited by 1
  11. 2025SynGraph: A Dynamic Graph-LLM Synthesis Framework for Sparse Streaming User Sentiment Modelingcited by 0
  12. 2026SemanticAgent: A Semantics-Aware Framework for Text-to-SQL Data Synthesiscited by 0