# Zero-Shot Information Extraction as a Unified Text-to-Triple Translation

arXiv: https://arxiv.org/abs/2109.11171 · HTML: https://mlsanity.com/paper/2109.11171/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [How Context Affects Language Models' Factual Predictions](https://mlsanity.com/paper/2005.04611/) (2020)
- [Language Models are Open Knowledge Graphs](https://mlsanity.com/paper/2010.11967/) (2020)
- [FewRel 2.0: Towards More Challenging Few-Shot Relation Classification](https://mlsanity.com/paper/1910.07124/) (2019)
- [Multi2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT](https://mlsanity.com/paper/2009.08128/) (2020)
- [Getting More Out Of Syntax with PropS](https://mlsanity.com/paper/1603.01648/) (2016)
- [Efficient long-distance relation extraction with DG-SpanBERT](https://mlsanity.com/paper/2004.03636/) (2020)

## Built on by (most cited first, 22 of 22)

- [A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models](https://mlsanity.com/paper/2501.13958/) (2025)
- [Universal Information Extraction as Unified Semantic Matching](https://mlsanity.com/paper/2301.03282/) (2023)
- [DeepStruct: Pretraining of Language Models for Structure Prediction](https://mlsanity.com/paper/2205.10475/) (2022)
- [SAC-KG: Exploiting Large Language Models as Skilled Automatic Constructors for Domain Knowledge Graphs](https://mlsanity.com/paper/2410.02811/) (2024)
- [CodeKGC: Code Language Model for Generative Knowledge Graph Construction](https://mlsanity.com/paper/2304.09048/) (2023)
- [Generative Prompt Tuning for Relation Classification](https://mlsanity.com/paper/2210.12435/) (2022)
- [PIVOINE: Instruction Tuning for Open-world Information Extraction](https://mlsanity.com/paper/2305.14898/) (2023)
- [A Survey on Open Information Extraction from Rule-based Model to Large Language Model](https://mlsanity.com/paper/2208.08690/) (2022)
- [Benchmarking Language Models for Code Syntax Understanding](https://mlsanity.com/paper/2210.14473/) (2022)
- [ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizations](https://mlsanity.com/paper/2203.13602/) (2022)
- [Zero-shot Triplet Extraction by Template Infilling](https://mlsanity.com/paper/2212.10708/) (2022)
- [Glitter or Gold? Deriving Structured Insights from Sustainability Reports via Large Language Models](https://mlsanity.com/paper/2310.05628/) (2023)
- [GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction](https://mlsanity.com/paper/2506.00649/) (2025)
- [PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion](https://mlsanity.com/paper/2210.13715/) (2022)
- [Statements: Universal Information Extraction from Tables with Large Language Models for ESG KPIs](https://mlsanity.com/paper/2406.19102/) (2024)
- [Mapping and Cleaning Open Commonsense Knowledge Bases with Generative Translation](https://mlsanity.com/paper/2306.12766/) (2023)
- [Beyond the Numbers: Transparency in Relation Extraction Benchmark Creation and Leaderboards](https://mlsanity.com/paper/2411.05224/) (2024)
- [IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models](https://mlsanity.com/paper/2210.14128/) (2022)
- [Enriching Relation Extraction with OpenIE](https://mlsanity.com/paper/2212.09376/) (2022)
- [CPTuning: Contrastive Prompt Tuning for Generative Relation Extraction](https://mlsanity.com/paper/2501.02196/) (2025)
- [InvertiTune: High-Quality Data Synthesis for Cost-Effective Single-Shot Text-to-Knowledge Graph Generation](https://mlsanity.com/paper/2512.03197/) (2025)
- [Construction of Knowledge Graph based on Language Model](https://mlsanity.com/paper/2604.19137/) (2026)

## Abstract (arXiv)

We cast a suite of information extraction tasks into a text-to-triple translation framework. Instead of solving each task relying on task-specific datasets and models, we formalize the task as a translation between task-specific input text and output triples. By taking the task-specific input, we enable a task-agnostic translation by leveraging the latent knowledge that a pre-trained language model has about the task. We further demonstrate that a simple pre-training task of predicting which relational information corresponds to which input text is an effective way to produce task-specific outputs. This enables the zero-shot transfer of our framework to downstream tasks. We study the zero-shot performance of this framework on open information extraction (OIE2016, NYT, WEB, PENN), relation classification (FewRel and TACRED), and factual probe (Google-RE and T-REx). The model transfers non-trivially to most tasks and is often competitive with a fully supervised method without the need for any task-specific training. For instance, we significantly outperform the F1 score of the supervised open information extraction without needing to use its training set.