mlsanityData to 7 Oct 2026

An Empirical Study of Validating Synthetic Data for Formula Generation

arXiv 2407.10657 ↗First posted 15 Jul 2024cs.CL cs.AIcited by 0

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Large language models (LLMs) can be leveraged to help with writing formulas in spreadsheets, but resources on these formulas are scarce, impacting both the base performance of pre-trained models and limiting the ability to fine-tune them. Given a corpus of formulas, we can use a(nother) model to generate synthetic natural language utterances for fine-tuning. However, it is important to validate whether the NL generated by the LLM is indeed accurate to be beneficial for fine-tuning. In this paper, we provide empirical results on the impact of validating these synthetic training examples with surrogate objectives that evaluate the accuracy of the synthetic annotations. We demonstrate that validation improves performance over raw data across four models (2 open and 2 closed weight). Interestingly, we show that although validation tends to prune more challenging examples, it increases the complexity of problems that models can solve after being fine-tuned on validated data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 16 of its 24 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

16 of 24 references.

References · 16

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2023LIMA: Less Is More for Alignmentcited by 1,080
  5. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  6. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  7. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  8. 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
  9. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  10. 2024MEGAnno+: A Human-LLM Collaborative Annotation Systemcited by 47
  11. 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
  12. 2024Automated Data Curation for Robust Language Model Fine-Tuningcited by 27
4 more
  1. 2023FLAME: A small language model for spreadsheet formulascited by 15
  2. 2023From Words to Code: Harnessing Data for Program Synthesis from Natural Languagecited by 11
  3. 2022CORNET: Learning Table Formatting Rules By Examplecited by 4
  4. 2024Solving Data-centric Tasks using Large Language Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs
  2. 2025Scaling Laws of Synthetic Data for Language Models
  3. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  4. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
  5. 2024Efficacy of Synthetic Data as a Benchmark
  6. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations
  7. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  8. 2024Evaluating Language Models as Synthetic Data Generators