mlsanityData to 14 Oct 2026

GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction

arXiv 2506.00649 ↗First posted 31 May 2025cs.CLcited by 4

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Information Extraction (IE) systems are traditionally domain-specific, requiring costly adaptation that involves expert schema design, data annotation, and model training. While Large Language Models have shown promise in zero-shot IE, performance degrades significantly in unseen domains where label definitions differ. This paper introduces GUIDEX, a novel method that automatically defines domain-specific schemas, infers guidelines, and generates synthetically labeled instances, allowing for better out-of-domain generalization. Fine-tuning Llama 3.1 with GUIDEX sets a new state-of-the-art across seven zeroshot Named Entity Recognition benchmarks. Models trained with GUIDEX gain up to 7 F1 points over previous methods without humanlabeled data, and nearly 2 F1 points higher when combined with it. Models trained on GUIDEX demonstrate enhanced comprehension of complex, domain-specific annotation schemas. Code, models, and synthetic datasets are available at neilus03.github.io/guidex.com

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 28 of its 52 references that are in the map. First posted on arXiv 31 May 2025.

Builds on

28 of 52 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  7. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  8. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  9. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  10. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 291
  11. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  12. 2023Large Language Models for Generative Information Extraction: A Surveycited by 185
16 more
  1. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 170
  2. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 119
  3. 2020CrossNER: Evaluating Cross-Domain Named Entity Recognitioncited by 104
  4. 2023Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extractioncited by 89
  5. 2021Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extractioncited by 73
  6. 2023CodeIE: Large Code Generation Models are Better Few-Shot Information Extractorscited by 68
  7. 2023Universal Information Extraction as Unified Semantic Matchingcited by 48
  8. 2024KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extractioncited by 34
  9. 2023Schema-Driven Information Extraction from Heterogeneous Tablescited by 25
  10. 2023Guideline Learning for In-context Information Extractioncited by 24
  11. 2024ADELIE: Aligning Large Language Models on Information Extractioncited by 24
  12. 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 23
  13. 2021Zero-Shot Information Extraction as a Unified Text-to-Triple Translationcited by 22
  14. 2020Denoising Relation Extraction from Document-level Distant Supervisioncited by 19
  15. 2023InstructIE: A Bilingual Instruction-based Information Extraction Datasetcited by 9
  16. 2023PromptRE: Weakly-Supervised Document-Level Relation Extraction via Prompting-Based Data Programmingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025FlexDoc: Parameterized Sampling for Diverse Multilingual Synthetic Documents for Training Document Understanding Modelscited by 2
  2. 2026SAM-NER: Semantic Archetype Mediation for Zero-Shot Named Entity Recognitioncited by 0
  3. 2026One Prompt Does Not Fit All: Self-Meta-Evolve for Personalized Information Extractioncited by 0
  4. 2026Calibrated Typed Decisions for Entity, Relation, and Event Extractioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models
  2. 2024A synthetic data approach for domain generalization of NLI models
  3. 2025A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition
  4. 2025FiNERweb: Datasets and Artifacts for Scalable Multilingual Named Entity Recognition
  5. 2024Rethinking Negative Instances for Generative Named Entity Recognition
  6. 2024Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative Study
  7. 2026SMADE-IE: Sparse Multi-Agent Framework with Evidence-Driven Debate for Zero-Shot Information Extraction
  8. 2025DocIE@XLLM25: In-Context Learning for Information Extraction using Fully Synthetic Demonstrations