mlsanityData to 7 Oct 2026

Structured Prediction as Translation between Augmented Natural Languages

arXiv 2101.05779 ↗First posted 14 Jan 2021cs.LG cs.CLcited by 183

Not on the map. Why?

Abstract (arXiv)

We propose a new framework, Translation between Augmented Natural Languages (TANL), to solve many structured prediction language tasks including joint entity and relation extraction, nested named entity recognition, relation classification, semantic role labeling, event extraction, coreference resolution, and dialogue state tracking. Instead of tackling the problem by training task-specific discriminative classifiers, we frame it as a translation task between augmented natural languages, from which the task-relevant information can be easily extracted. Our approach can match or outperform task-specific models on all tasks, and in particular, achieves new state-of-the-art results on joint entity and relation extraction (CoNLL04, ADE, NYT, and ACE2005 datasets), relation classification (FewRel and TACRED), and semantic role labeling (CoNLL-2005 and CoNLL-2012). We accomplish this while using the same architecture and hyperparameters for all tasks and even when training a single model to solve all tasks at the same time (multi-task learning). Finally, we show that our framework can also significantly improve the performance in a low-resource regime, thanks to better use of label semantics.

Key facts. Cited by 183 papers in the map (data through 7 Oct 2026). Builds on 47 of its 79 references that are in the map. First posted on arXiv 14 Jan 2021.

Builds on

47 of 79 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2016Matching Networks for One Shot Learningcited by 2,246
  8. 2020Reformer: The Efficient Transformercited by 1,658
  9. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  10. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  11. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  12. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
35 more
  1. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  2. 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 271
  3. 2019Entity, Relation, and Event Extraction with Contextualized Span Representationscited by 261
  4. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  5. 2017End-to-end Neural Coreference Resolutioncited by 240
  6. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  7. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
  8. 2019A Unified MRC Framework for Named Entity Recognitioncited by 199
  9. 2019Dice Loss for Data-imbalanced NLP Taskscited by 180
  10. 2019Simple BERT Models for Relation Extraction and Semantic Role Labelingcited by 154
  11. 2018Higher-order Coreference Resolution with Coarse-to-fine Inferencecited by 150
  12. 2019Span-based Joint Entity and Relation Extraction with Transformer Pre-trainingcited by 132
  13. 2020Exploring and Predicting Transferability across NLP Taskscited by 126
  14. 2019BERT for Coreference Resolution: Baselines and Analysiscited by 125
  15. 2016Recurrent Neural Network Grammarscited by 109
  16. 2019Entity-Relation Extraction as Multi-Turn Question Answeringcited by 96
  17. 2019FewRel 2.0: Towards More Challenging Few-Shot Relation Classificationcited by 94
  18. 2019A General Framework for Information Extraction using Dynamic Span Graphscited by 93
  19. 2019Coreference Resolution as Query-based Span Predictioncited by 91
  20. 2019Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extractioncited by 73
  21. 2020SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teachingcited by 71
  22. 2014Grammar as a Foreign Languagecited by 70
  23. 2020Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsingcited by 59
  24. 2019Neural Architectures for Nested NER through Linearizationcited by 56
  25. 2015Transition-Based Dependency Parsing with Stack Long Short-Term Memorycited by 45
  26. 2019Cloze-driven Pretraining of Self-attention Networkscited by 41
  27. 2019Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategycited by 41
  28. 2018One for All: Neural Joint Modeling of Entities and Eventscited by 37
  29. 2019Nested Named Entity Recognition via Second-best Sequence Learning and Decodingcited by 34
  30. 2020Augmented Natural Language for Generative Sequence Labelingcited by 29
  31. 2019Dependency or Span, End-to-End Uniform Semantic Role Labelingcited by 22
  32. 2020Beyond [CLS] through Ranking by Generationcited by 20
  33. 2018Multi-Task Learning for Sequence Tagging: An Empirical Studycited by 15
  34. 2020Downstream Model Design of Pre-trained Language Model for Relation Extraction Taskcited by 7
  35. 2020Efficient long-distance relation extraction with DG-SpanBERTcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 50; 2023: 59; 2024: 30; 2025: 15; 2026: 10.

183 papers.

Sort:

By area:

  1. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  3. 2022Unified Structure Generation for Universal Information Extractioncited by 220
  4. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  5. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  6. 2023Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulnesscited by 130
  7. 2021Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extractioncited by 129
  8. 2023Revisiting Relation Extraction in the era of Large Language Modelscited by 100
  9. 2021Aspect Sentiment Quad Prediction as Paraphrase Generationcited by 99
  10. 2023Do We Still Need Clinical Language Models?cited by 97
  11. 2021DEGREE: A Data-Efficient Generation-Based Event Extraction Modelcited by 82
  12. 2022Code4Struct: Code Generation for Few-Shot Event Structure Predictioncited by 70
38 more
  1. 2021Tailor: Generating and Perturbing Text with Semantic Controlscited by 67
  2. 2023CodeIE: Large Code Generation Models are Better Few-Shot Information Extractorscited by 67
  3. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  4. 2022RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extractioncited by 60
  5. 2022Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extractioncited by 54
  6. 2022Generative Knowledge Graph Construction: A Reviewcited by 49
  7. 2023MvP: Multi-view Prompting Improves Aspect Sentiment Tuple Predictioncited by 49
  8. 2021Text-to-Table: A New Way of Information Extractioncited by 47
  9. 2022DeepStruct: Pretraining of Language Models for Structure Predictioncited by 45
  10. 2022Autoregressive Structured Prediction with Language Modelscited by 43
  11. 2022Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extractioncited by 40
  12. 2021Paradigm Shift in Natural Language Processingcited by 39
  13. 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 39
  14. 2021Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NERcited by 38
  15. 2022BigBIO: A Framework for Data-Centric Biomedical Natural Language Processingcited by 36
  16. 2022Dynamic Prefix-Tuning for Generative Template-based Event Extractioncited by 35
  17. 2021Document-level Entity-based Extraction as Template Generationcited by 33
  18. 2022Summarization as Indirect Supervision for Relation Extractioncited by 31
  19. 2021ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoningcited by 30
  20. 2023How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?cited by 30
  21. 2022Improving Aspect Sentiment Quad Prediction via Template-Order Data Augmentationcited by 29
  22. 2023CodeKGC: Code Language Model for Generative Knowledge Graph Constructioncited by 28
  23. 2023TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extractioncited by 27
  24. 2022GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Rolescited by 25
  25. 2024Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labelingcited by 24
  26. 2021Zero-Shot Information Extraction as a Unified Text-to-Triple Translationcited by 22
  27. 2022Label Semantics for Few Shot Named Entity Recognitioncited by 21
  28. 2023A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontierscited by 21
  29. 2023Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuningcited by 21
  30. 2023Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captionscited by 21
  31. 2022DICE: Data-Efficient Clinical Event Extraction with Generative Modelscited by 20
  32. 2022Parallel Instance Query Network for Named Entity Recognitioncited by 19
  33. 2023AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Modelcited by 19
  34. 2024Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trendscited by 19
  35. 2022Label Semantic Aware Pre-training for Few-shot Text Classificationcited by 18
  36. 2022Unified Semantic Typing with Meaningful Label Inferencecited by 18
  37. 2022Retrieval-Augmented Generative Question Answering for Event Argument Extractioncited by 18
  38. 2021Translate & Fill: Improving Zero-Shot Multilingual Semantic Parsing with Synthetic Datacited by 17

50 most cited of 183 All 183 (JSON)