# Investigating Cost-Efficiency of LLM-Generated Training Data for Conversational Semantic Frame Analysis

arXiv: https://arxiv.org/abs/2410.06550 · HTML: https://mlsanity.com/paper/2410.06550/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 9 Oct 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!](https://mlsanity.com/paper/2303.08559/) (2023)
- [Pushing the Limits of ChatGPT on NLP Tasks](https://mlsanity.com/paper/2306.09719/) (2023)
- [Open-Domain Frame Semantic Parsing Using Transformers](https://mlsanity.com/paper/2010.10998/) (2020)
- [JaMIE: A Pipeline Japanese Medical Information Extraction System](https://mlsanity.com/paper/2111.04261/) (2021)

## Built on by (most cited first, 1 of 1)

- [Breaking the Silence: A Dataset and Benchmark for Bangla Text-to-Gloss Translation](https://mlsanity.com/paper/2504.02293/) (2025)

## Abstract (arXiv)

Recent studies have demonstrated that few-shot learning allows LLMs to generate training data for supervised models at a low cost. However, the quality of LLM-generated data may not entirely match that of human-labeled data. This raises a crucial question: how should one balance the trade-off between the higher quality but more expensive human data and the lower quality yet substantially cheaper LLM-generated data? In this paper, we synthesized training data for conversational semantic frame analysis using GPT-4 and examined how to allocate budgets optimally to achieve the best performance. Our experiments, conducted across various budget levels, reveal that optimal cost-efficiency is achieved by combining both human and LLM-generated data across a wide range of budget levels. Notably, as the budget decreases, a higher proportion of LLM-generated data becomes more preferable.