mlsanityData to 14 Oct 2026

Universal Information Extraction with Meta-Pretrained Self-Retrieval

arXiv 2306.10444 ↗First posted 18 Jun 2023cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Universal Information Extraction~(Universal IE) aims to solve different extraction tasks in a uniform text-to-structure generation manner. Such a generation procedure tends to struggle when there exist complex information structures to be extracted. Retrieving knowledge from external knowledge bases may help models to overcome this problem but it is impossible to construct a knowledge base suitable for various IE tasks. Inspired by the fact that large amount of knowledge are stored in the pretrained language models~(PLM) and can be retrieved explicitly, in this paper, we propose MetaRetriever to retrieve task-specific knowledge from PLMs to enhance universal IE. As different IE tasks need different knowledge, we further propose a Meta-Pretraining Algorithm which allows MetaRetriever to quicktly achieve maximum task-specific retrieval performance when fine-tuning on downstream IE tasks. Experimental results show that MetaRetriever achieves the new state-of-the-art on 4 IE tasks, 12 datasets under fully-supervised, low-resource and few-shot scenarios.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 31 of its 51 references that are in the map. First posted on arXiv 18 Jun 2023.

Builds on

31 of 51 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  9. 2016Matching Networks for One Shot Learningcited by 2,255
  10. 2019Language Models as Knowledge Bases?cited by 1,936
  11. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
19 more
  1. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  2. 2016Neural Architectures for Named Entity Recognitioncited by 550
  3. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  4. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 322
  5. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  6. 2019Entity, Relation, and Event Extraction with Contextualized Span Representationscited by 262
  7. 2018Probabilistic Model-Agnostic Meta-Learningcited by 256
  8. 2022Unified Structure Generation for Universal Information Extractioncited by 221
  9. 2020Event Extraction by Answering (Almost) Natural Questionscited by 177
  10. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  11. 2017Joint Extraction of Entities and Relations Based on a Novel Tagging Schemecited by 102
  12. 2021A Unified Generative Framework for Aspect-Based Sentiment Analysiscited by 97
  13. 2021Improving Named Entity Recognition by External Context Retrieving and Cooperative Learningcited by 76
  14. 2019Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networkscited by 33
  15. 2021A Survey on Deep Learning Event Extraction: Approaches and Applicationscited by 12
  16. 2018Nugget Proposal Networks for Chinese Event Detectioncited by 11
  17. 2021Reinforcement Learning-based Dialogue Guided Event Extraction to Exploit Argument Relationscited by 10
  18. 2022TURNER: The Uncertainty-based Retrieval Framework for Chinese NERcited by 4
  19. 2022Type Information Utilized Event Detection via Multi-Channel GNNs in Electrical Power Systemscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
  2. 2024SportQA: A Benchmark for Sports Understanding in Large Language Modelscited by 15
  3. 2025U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMscited by 10
  4. 2025Label Drop for Multi-Aspect Relation Modeling in Universal Information Extractioncited by 1
  5. 2026Mitigating Entity Type Confusion in Cross-Domain NER via Multidimensional Quantification and Reasoning Enhancementcited by 0