mlsanityData to 14 Oct 2026

CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors

arXiv 2305.05711 ↗First posted 9 May 2023cs.CL cs.AIcited by 68

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) pre-trained on massive corpora have demonstrated impressive few-shot learning ability on many NLP tasks. A common practice is to recast the task into a text-to-text format such that generative LLMs of natural language (NL-LLMs) like GPT-3 can be prompted to solve it. However, it is nontrivial to perform information extraction (IE) tasks with NL-LLMs since the output of the IE task is usually structured and therefore is hard to be converted into plain text. In this paper, we propose to recast the structured output in the form of code instead of natural language and utilize generative LLMs of code (Code-LLMs) such as Codex to perform IE tasks, in particular, named entity recognition and relation extraction. In contrast to NL-LLMs, we show that Code-LLMs can be well-aligned with these IE tasks by designing code-style prompts and formulating these IE tasks as code generation tasks. Experiment results on seven benchmarks show that our method consistently outperforms fine-tuning moderate-size pre-trained models specially designed for IE tasks (e.g., UIE) and prompting NL-LLMs under few-shot settings. We further conduct a series of in-depth analyses to demonstrate the merits of leveraging Code-LLMs for IE tasks.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 22 of its 44 references that are in the map. First posted on arXiv 9 May 2023.

Builds on

22 of 44 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  10. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 322
  11. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 256
  12. 2022Binding Language Models in Symbolic Languagescited by 228
10 more
  1. 2022Unified Structure Generation for Universal Information Extractioncited by 221
  2. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 207
  3. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  4. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  5. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  6. 2021A Unified Generative Framework for Aspect-Based Sentiment Analysiscited by 97
  7. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  8. 2021GenIE: Generative Information Extractioncited by 53
  9. 2021UniRE: A Unified Label Space for Entity Relation Extractioncited by 38
  10. 2020A Frustratingly Easy Approach for Entity and Relation Extractioncited by 37

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 14; 2024: 23; 2025: 21; 2026: 10.

68 papers.

Sort:

By area:

  1. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  2. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  3. 2023Large Language Models for Generative Information Extraction: A Surveycited by 185
  4. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 119
  5. 2024Large Language Models Meet NLP: A Surveycited by 103
  6. 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
  7. 2024CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Modelscited by 76
  8. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  9. 2023Statler: State-Maintaining Language Models for Embodied Reasoningcited by 40
  10. 2023Retrieval-Augmented Code Generation for Universal Information Extractioncited by 40
  11. 2024KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extractioncited by 34
  12. 2024ADELIE: Aligning Large Language Models on Information Extractioncited by 24
38 more
  1. 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 23
  2. 2023When Do Program-of-Thoughts Work for Reasoning?cited by 19
  3. 2025Instruction-Tuning LLMs for Event Extraction with Annotation Guidelinescited by 18
  4. 2025GraphCodeAgent: Dual Graph-Guided LLM Agent for Retrieval-Augmented Repo-Level Code Generationcited by 17
  5. 2023Self-Improving for Zero-Shot Named Entity Recognition with Large Language Modelscited by 15
  6. 2023Code-Style In-Context Learning for Knowledge-Based Question Answeringcited by 13
  7. 2024CodeTaxo: Enhancing Taxonomy Expansion with Limited Examples via Code Language Promptscited by 10
  8. 2023InstructIE: A Bilingual Instruction-based Information Extraction Datasetcited by 9
  9. 2024On-the-fly Definition Augmentation of LLMs for Biomedical NERcited by 9
  10. 2024LTNER: Large Language Model Tagging for Named Entity Recognition with Contextualized Entity Markingcited by 7
  11. 2023Benchmarking Large Language Models with Augmented Instructions for Fine-grained Information Extractioncited by 6
  12. 2023Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT -- A Text-to-SQL Parsing Comparisoncited by 6
  13. 2024On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generationcited by 6
  14. 2024DAC: Decomposed Automation Correction for Text-to-SQLcited by 6
  15. 2025Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extractioncited by 6
  16. 2025MR-UIE: Multi-Perspective Reasoning with Reinforcement Learning for Universal Information Extractioncited by 6
  17. 2024IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpuscited by 5
  18. 2024Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Modelscited by 4
  19. 2025GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extractioncited by 4
  20. 2025Multilingual Dialogue Generation and Localization with Dialogue Act Scriptingcited by 4
  21. 2024Embedded Named Entity Recognition using Probing Classifierscited by 3
  22. 2024RUIE: Retrieval-based Unified Information Extraction using Large Language Modelcited by 3
  23. 2024KnowCoder-X: Boosting Multilingual Information Extraction via Codecited by 3
  24. 2025Selecting and Merging: Towards Adaptable and Scalable Named Entity Recognition with Large Language Modelscited by 3
  25. 2023Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Modelscited by 2
  26. 2024RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructionscited by 2
  27. 2024Are Human Rules Necessary? Generating Reusable APIs with CoT Reasoning and In-Context Learningcited by 2
  28. 2025Towards Event Extraction with Massive Types: LLM-based Collaborative Annotation and Partitioning Extractioncited by 2
  29. 2025MPL: Multiple Programming Languages with Large Language Models for Information Extractioncited by 2
  30. 2025EL4NER: Ensemble Learning for Named Entity Recognition via Multiple Small-Parameter Large Language Modelscited by 2
  31. 2026HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategycited by 2
  32. 2024Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstructioncited by 1
  33. 2024Is There a One-Model-Fits-All Approach to Information Extraction? Revisiting Task Definition Biasescited by 1
  34. 2024Multi-tool Integration Application for Math Reasoning Using Large Language Modelcited by 1
  35. 2025Biomedical Relation Extraction via Adaptive Document-Relation Cross-Mapping and Concept Unique Identifiercited by 1
  36. 2025EMRModel: A Large Language Model for Extracting Medical Consultation Dialogues into Structured Medical Recordscited by 1
  37. 2025KnowCoder-V2: Deep Knowledge Analysiscited by 1
  38. 2025A Semantic Parsing Framework for End-to-End Time Normalizationcited by 1

50 most cited of 68 All 68 (JSON)