mlsanityData to 7 Oct 2026

Synthetic Data Generation with Large Language Models for Personalized Community Question Answering

arXiv 2410.22182 ↗First posted 29 Oct 2024cs.IRcited by 8

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Personalization in Information Retrieval (IR) is a topic studied by the research community since a long time. However, there is still a lack of datasets to conduct large-scale evaluations of personalized IR; this is mainly due to the fact that collecting and curating high-quality user-related information requires significant costs and time investment. Furthermore, the creation of datasets for Personalized IR (PIR) tasks is affected by both privacy concerns and the need for accurate user-related data, which are often not publicly available. Recently, researchers have started to explore the use of Large Language Models (LLMs) to generate synthetic datasets, which is a possible solution to generate data for low-resource tasks. In this paper, we investigate the potential of Large Language Models (LLMs) for generating synthetic documents to train an IR system for a Personalized Community Question Answering task. To study the effectiveness of IR models fine-tuned on LLM-generated data, we introduce a new dataset, named Sy-SE-PQA. We build Sy-SE-PQA based on an existing dataset, SE-PQA, which consists of questions and answers posted on the popular StackExchange communities. Starting from questions in SE-PQA, we generate synthetic answers using different prompt techniques and LLMs. Our findings suggest that LLMs have high potential in generating data tailored to users' needs. The synthetic data can replace human-written training data, even if the generated data may contain incorrect information.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 30 of its 52 references that are in the map. First posted on arXiv 29 Oct 2024.

Builds on

30 of 52 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personas28 Jun
  2. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations11 Oct

Other references · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  10. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  11. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  12. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
16 more
  1. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  2. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  3. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  4. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  5. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  6. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 121
  7. 2020Training Question Answering Models From Synthetic Datacited by 95
  8. 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89
  9. 2022Data Augmentation for Intent Classification with Off-the-shelf Large Language Modelscited by 49
  10. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
  11. 2022Weakly Supervised Data Augmentation Through Prompting for Dialogue Understandingcited by 32
  12. 2023On the Design of AI-powered Code Assistants for Notebookscited by 28
  13. 2022ClueWeb22: 10 Billion Web Documents with Visual and Semantic Informationcited by 19
  14. 2024TREC iKAT 2023: The Interactive Knowledge Assistance Track Overviewcited by 15
  15. 2023SE-PQA: Personalized Community Question Answeringcited by 7
  16. 2017Ensemble Methods for Personalized E-Commerce Search Challenge at CIKM Cup 2016cited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025Investigating Task Arithmetic for Zero-Shot Information Retrievalcited by 4
  2. 2026SITUATE -- Synthetic Object Counting Dataset for VLM trainingcited by 2
  3. 2026BioGraphletQA: Knowledge-Anchored Generation of Complex QA Datasetscited by 1
  4. 2025Unlocking the Potential of Large Language Models in the Nuclear Industry with Synthetic Datacited by 0
  5. 2025Investigating Large Language Models' Linguistic Abilities for Text Preprocessingcited by 0
  6. 2026Fine-tuning Small Language Models as Efficient Enterprise Search Relevance Labelerscited by 0
  7. 2026DIETA: A Decoder-only transformer-based model for Italian-English machine TrAnslationcited by 0
  8. 2026ChildEval: When large language models meet children's personalitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Personalizing Large Language Models using Retrieval Augmented Generation and Knowledge Graph
  2. 2024Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models
  3. 2026Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculations
  4. 2024Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation
  5. 2024Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs
  6. 2025BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback
  7. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  8. 2024CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation