mlsanityData to 7 Oct 2026

Large Language Models for Data Annotation and Synthesis: A Survey

arXiv 2402.13446 ↗First posted 21 Feb 2024cs.CLcited by 201

Not on the map. Why?

Cited by 28 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Data annotation and synthesis generally refers to the labeling or generating of raw data with relevant information, which could be used for improving the efficacy of machine learning models. The process, however, is labor-intensive and costly. The emergence of advanced Large Language Models (LLMs), exemplified by GPT-4, presents an unprecedented opportunity to automate the complicated process of data annotation and synthesis. While existing surveys have extensively covered LLM architecture, training, and general applications, we uniquely focus on their specific utility for data annotation. This survey contributes to three core aspects: LLM-Based Annotation Generation, LLM-Generated Annotations Assessment, and LLM-Generated Annotations Utilization. Furthermore, this survey includes an in-depth taxonomy of data types that LLMs can annotate, a comprehensive review of learning strategies for models utilizing LLM-generated annotations, and a detailed discussion of the primary challenges and limitations associated with using LLMs for data annotation and synthesis. Serving as a key guide, this survey aims to assist researchers and practitioners in exploring the potential of the latest LLMs for data annotation, thereby fostering future advancements in this critical field.

Key facts. Cited by 201 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Feb 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 108; 2026: 68.

201 papers.

Sort:

By area:

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  3. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  4. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  5. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 119
  6. 2025The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMscited by 72
  7. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRcited by 53
  8. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  9. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 39
  10. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  11. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 35
  12. 2024ClinicalBench: Can LLMs Beat Traditional ML Models in Clinical Prediction?cited by 31
38 more
  1. 2025Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challengescited by 31
  2. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  3. 2025DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Modelscited by 29
  4. 2024OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domaincited by 28
  5. 2025AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Predictioncited by 25
  6. 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoningcited by 25
  7. 2024How to Synthesize Text Data without Model Collapse?cited by 24
  8. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  9. 2024From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Taskscited by 22
  10. 2026SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Modelscited by 20
  11. 2024Can LLMs Convert Graphs to Text-Attributed Graphs?cited by 19
  12. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  13. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundarycited by 17
  14. 2025Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absentcited by 15
  15. 2025Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Modelscited by 15
  16. 2025Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillationcited by 15
  17. 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 15
  18. 2025Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimizationcited by 14
  19. 2026ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agentscited by 13
  20. 2026Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theorycited by 13
  21. 2024On Limitations of LLM as Annotator for Low Resource Languagescited by 12
  22. 2025Graph Neural Networks in Modern AI-aided Drug Discoverycited by 12
  23. 2024LLM Chain Ensembles for Scalable and Accurate Data Annotationcited by 11
  24. 2024Beyond Math: Stories as a Testbed for Memorization-Constrained Reasoning in LLMscited by 11
  25. 2025Understanding Impact of Human Feedback via Influence Functionscited by 11
  26. 2026Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillationcited by 11
  27. 2025Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Datacited by 10
  28. 2025Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Abilitycited by 10
  29. 2025PromptPex: Automatic Test Generation for Language Model Promptscited by 9
  30. 2025Large Language Models Meet Legal Artificial Intelligence: A Surveycited by 9
  31. 2025A Survey on Efficient Large Language Model Training: From Data-centric Perspectivescited by 9
  32. 2025Benchmarking Multi-Step Legal Reasoning and Analyzing Chain-of-Thought Effects in Large Language Modelscited by 9
  33. 2026Triplets Better Than Pairs: Towards Stable and Effective Self-Play Fine-Tuning for LLMscited by 9
  34. 2026Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMscited by 9
  35. 2024A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLIcited by 8
  36. 2025AI vs. Human Moderators: A Comparative Evaluation of Multimodal LLMs in Content Moderation for Brand Safetycited by 8
  37. 2026Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarioscited by 8
  38. 2024Rethinking Emotion Annotations in the Era of Large Language Modelscited by 7

50 most cited of 201 All 201 (JSON)