# Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractors

arXiv: https://arxiv.org/abs/2404.17807 · HTML: https://mlsanity.com/paper/2404.17807/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 64 references that are in the map.
- First posted on arXiv 27 Apr 2024.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [Meta-learning via Language Model In-context Tuning](https://mlsanity.com/paper/2110.07814/) (2021)
- [Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!](https://mlsanity.com/paper/2303.08559/) (2023)
- [Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders](https://mlsanity.com/paper/2010.03851/) (2020)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction](https://mlsanity.com/paper/2109.03659/) (2021)
- [GPT-RE: In-context Learning for Relation Extraction using Large Language Models](https://mlsanity.com/paper/2305.02105/) (2023)
- [RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction](https://mlsanity.com/paper/2203.09101/) (2022)
- [A Hierarchical Framework for Relation Extraction with Reinforcement Learning](https://mlsanity.com/paper/1811.03925/) (2018)
- [Relation Classification via Recurrent Neural Network](https://mlsanity.com/paper/1508.01006/) (2015)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)

## Built on by (most cited first, 13 of 13)

- [GLiREL -- Generalist Model for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2501.03172/) (2025)
- [Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey](https://mlsanity.com/paper/2504.14520/) (2025)
- [GLiDRE: Generalist Lightweight model for Document-level Relation Extraction](https://mlsanity.com/paper/2508.00757/) (2025)
- [Delta-WKV: A Novel Meta-in-Context Learner for MRI Super-Resolution](https://mlsanity.com/paper/2502.20852/) (2025)
- [GLiNER-Relex: A Unified Framework for Joint Named Entity Recognition and Relation Extraction](https://mlsanity.com/paper/2605.10108/) (2026)
- [PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents](https://mlsanity.com/paper/2605.10341/) (2026)
- [Prompt Segmentation and Annotation Optimisation: Controlling LLM Behaviour via Optimised Segment-Level Annotations](https://mlsanity.com/paper/2605.14561/) (2026)
- [Software Defined Vehicle Code Generation: A Few-Shot Prompting Approach](https://mlsanity.com/paper/2511.04849/) (2025)
- [AsynDBT: Asynchronous Distributed Bilevel Tuning for efficient In-Context Learning with Large Language Models](https://mlsanity.com/paper/2602.17694/) (2026)
- [Design-Specification Tiling for ICL-based CAD Code Generation](https://mlsanity.com/paper/2603.12712/) (2026)
- [A Cross-graph Tuning-free GNN Prompting Framework](https://mlsanity.com/paper/2604.00399/) (2026)
- [Learn-To-Learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLM](https://mlsanity.com/paper/2605.01973/) (2026)
- [Meta-Learning Preferences for Multilingual LLM Alignment](https://mlsanity.com/paper/2607.13315/) (2026)

## Abstract (arXiv)

Relation extraction (RE) is an important task that aims to identify the relationships between entities in texts. While large language models (LLMs) have revealed remarkable in-context learning (ICL) capability for general zero and few-shot learning, recent studies indicate that current LLMs still struggle with zero and few-shot RE. Previous studies are mainly dedicated to design prompt formats and select good examples for improving ICL-based RE. Although both factors are vital for ICL, if one can fundamentally boost the ICL capability of LLMs in RE, the zero and few-shot RE performance via ICL would be significantly improved. To this end, we introduce textscMicre (Meta In-Context learning of LLMs for Relation Extraction), a new meta-training framework for zero and few-shot RE where an LLM is tuned to do ICL on a diverse collection of RE datasets (i.e., learning to learn in context for RE). Through meta-training, the model becomes more effectively to learn a new RE task in context by conditioning on a few training examples with no parameter updates or task-specific templates at inference time, enabling better zero and few-shot task generalization. We experiment textscMicre on various LLMs with different model scales and 12 public RE datasets, and then evaluate it on unseen RE benchmarks under zero and few-shot settings. textscMicre delivers comparable or superior performance compared to a range of baselines including supervised fine-tuning and typical in-context learning methods. We find that the gains are particular significant for larger model scales, and using a diverse set of the meta-training RE datasets is key to improvements. Empirically, we show that textscMicre can transfer the relation semantic knowledge via relation label name during inference on target RE datasets.