# Augmented Natural Language for Generative Sequence Labeling

arXiv: https://arxiv.org/abs/2009.13272 · HTML: https://mlsanity.com/paper/2009.13272/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 29 of 29)

- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning](https://mlsanity.com/paper/2305.13971/) (2023)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Aspect Sentiment Quad Prediction as Paraphrase Generation](https://mlsanity.com/paper/2110.00796/) (2021)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [DeepStruct: Pretraining of Language Models for Structure Prediction](https://mlsanity.com/paper/2205.10475/) (2022)
- [Autoregressive Structured Prediction with Language Models](https://mlsanity.com/paper/2210.14698/) (2022)
- [Controllable Semantic Parsing via Retrieval Augmentation](https://mlsanity.com/paper/2110.08458/) (2021)
- [An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling](https://mlsanity.com/paper/2109.13023/) (2021)
- [DiffuSum: Generation Enhanced Extractive Summarization with Diffusion](https://mlsanity.com/paper/2305.01735/) (2023)
- [Decomposed Meta-Learning for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2204.05751/) (2022)
- [Label Semantics for Few Shot Named Entity Recognition](https://mlsanity.com/paper/2203.08985/) (2022)
- [Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning](https://mlsanity.com/paper/2311.03748/) (2023)
- [Label Semantic Aware Pre-training for Few-shot Text Classification](https://mlsanity.com/paper/2204.07128/) (2022)
- [Transforming Sequence Tagging Into A Seq2Seq Task](https://mlsanity.com/paper/2203.08378/) (2022)
- [On the Evolution of Knowledge Graphs: A Survey and Perspective](https://mlsanity.com/paper/2310.04835/) (2023)
- [Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-based Encoder](https://mlsanity.com/paper/2109.04500/) (2021)
- [DiffusionNER: Boundary Diffusion for Named Entity Recognition](https://mlsanity.com/paper/2305.13298/) (2023)
- [Distillation of encoder-decoder transformers for sequence labelling](https://mlsanity.com/paper/2302.05454/) (2023)
- [Sequence-to-Sequence Models for Extracting Information from Registration and Legal Documents](https://mlsanity.com/paper/2201.05658/) (2022)
- [Confidence Calibration for Intent Detection via Hyperspherical Space and Rebalanced Accuracy-Uncertainty Loss](https://mlsanity.com/paper/2203.09278/) (2022)
- [Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models](https://mlsanity.com/paper/2110.07331/) (2021)
- [GROOT: Corrective Reward Optimization for Generative Sequential Labeling](https://mlsanity.com/paper/2209.14694/) (2022)
- [How Does Beam Search improve Span-Level Confidence Estimation in Generative Sequence Labeling?](https://mlsanity.com/paper/2212.10767/) (2022)
- [Unifying Token and Span Level Supervisions for Few-Shot Sequence Labeling](https://mlsanity.com/paper/2307.07946/) (2023)
- [Joint Text and Label Generation for Spoken Language Understanding](https://mlsanity.com/paper/2105.05052/) (2021)
- [Causal Interventions-based Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2305.01914/) (2023)
- [Exploration of Marker-Based Approaches in Argument Mining through Augmented Natural Language](https://mlsanity.com/paper/2406.08606/) (2024)

## Abstract (arXiv)

We propose a generative framework for joint sequence labeling and sentence-level classification. Our model performs multiple sequence labeling tasks at once using a single, shared natural language output space. Unlike prior discriminative methods, our model naturally incorporates label semantics and shares knowledge across tasks. Our framework is general purpose, performing well on few-shot, low-resource, and high-resource tasks. We demonstrate these advantages on popular named entity recognition, slot labeling, and intent classification benchmarks. We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot (75.0% rightarrow 90.9%) and 1-shot (70.4% rightarrow 81.0%) state-of-the-art results. Furthermore, our model generates large improvements (46.27% rightarrow 63.83%) in low-resource slot labeling over a BERT baseline by incorporating label semantics. We also maintain competitive results on high-resource tasks, performing within two points of the state-of-the-art on all tasks and setting a new state-of-the-art on the SNIPS dataset.