mlsanityData to 7 Oct 2026

Investigating Cost-Efficiency of LLM-Generated Training Data for Conversational Semantic Frame Analysis

arXiv 2410.06550 ↗First posted 9 Oct 2024cs.CL cs.AIcited by 1

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Recent studies have demonstrated that few-shot learning allows LLMs to generate training data for supervised models at a low cost. However, the quality of LLM-generated data may not entirely match that of human-labeled data. This raises a crucial question: how should one balance the trade-off between the higher quality but more expensive human data and the lower quality yet substantially cheaper LLM-generated data? In this paper, we synthesized training data for conversational semantic frame analysis using GPT-4 and examined how to allocate budgets optimally to achieve the best performance. Our experiments, conducted across various budget levels, reveal that optimal cost-efficiency is achieved by combining both human and LLM-generated data across a wide range of budget levels. Notably, as the budget decreases, a higher proportion of LLM-generated data becomes more preferable.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 8 of its 20 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

8 of 20 references.

References · 8

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2022Is GPT-3 a Good Data Annotator?cited by 208
  4. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  5. 2023Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!cited by 119
  6. 2023Pushing the Limits of ChatGPT on NLP Taskscited by 27
  7. 2020Open-Domain Frame Semantic Parsing Using Transformerscited by 12
  8. 2021JaMIE: A Pipeline Japanese Medical Information Extraction Systemcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Breaking the Silence: A Dataset and Benchmark for Bangla Text-to-Gloss Translationcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings
  2. 2026Paying for Too Many Tokens? Valid and Cost-Efficient Multimodal LLM Annotation with Simple Heuristics
  3. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents
  4. 2026AgenticAI-DialogGen: Topic-Guided Conversation Generation for Fine-Tuning and Evaluating Short- and Long-Term Memories of LLMs
  5. 2024SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking
  6. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversation
  7. 2025Regularization Through Reasoning: Systematic Improvements in Language Model Classification via Explanation-Enhanced Fine-Tuning
  8. 2025Modular Techniques for Synthetic Long-Context Data Generation in Language Model Training and Evaluation