# Enhanced Language Representation with Label Knowledge for Span Extraction

arXiv: https://arxiv.org/abs/2111.00884 · HTML: https://mlsanity.com/paper/2111.00884/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Event Extraction by Answering (Almost) Natural Questions](https://mlsanity.com/paper/2004.13625/) (2020)
- [Named Entity Recognition as Dependency Parsing](https://mlsanity.com/paper/2005.07150/) (2020)
- [Entity-Relation Extraction as Multi-Turn Question Answering](https://mlsanity.com/paper/1905.05529/) (2019)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Chinese NER Using Lattice LSTM](https://mlsanity.com/paper/1805.02023/) (2018)
- [Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation](https://mlsanity.com/paper/1809.09078/) (2018)
- [Neural Architectures for Nested NER through Linearization](https://mlsanity.com/paper/1908.06926/) (2019)
- [Glyce: Glyph-vectors for Chinese Character Representations](https://mlsanity.com/paper/1901.10125/) (2019)
- [Fast and Accurate Entity Recognition with Iterated Dilated Convolutions](https://mlsanity.com/paper/1702.02098/) (2017)
- [Nested Named Entity Recognition via Second-best Sequence Learning and Decoding](https://mlsanity.com/paper/1909.02250/) (2019)
- [Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks](https://mlsanity.com/paper/1906.03783/) (2019)
- [Neural Segmental Hypergraphs for Overlapping Mention Recognition](https://mlsanity.com/paper/1810.01817/) (2018)
- [Reading the Manual: Event Extraction as Definition Comprehension](https://mlsanity.com/paper/1912.01586/) (2019)
- [Label-aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition](https://mlsanity.com/paper/1804.09021/) (2018)
- [Cost-sensitive Regularization for Label Confusion-aware Event Detection](https://mlsanity.com/paper/1906.06003/) (2019)

## Built on by (most cited first, 3 of 3)

- [PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks](https://mlsanity.com/paper/2210.08855/) (2022)
- [Large-Scale Label Interpretation Learning for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2403.14222/) (2024)
- [MRC-based Nested Medical NER with Co-prediction and Adaptive Pre-training](https://mlsanity.com/paper/2403.15800/) (2024)

## Abstract (arXiv)

Span extraction, aiming to extract text spans (such as words or phrases) from plain texts, is a fundamental process in Information Extraction. Recent works introduce the label knowledge to enhance the text representation by formalizing the span extraction task into a question answering problem (QA Formalization), which achieves state-of-the-art performance. However, QA Formalization does not fully exploit the label knowledge and suffers from low efficiency in training/inference. To address those problems, we introduce a new paradigm to integrate label knowledge and further propose a novel model to explicitly and efficiently integrate label knowledge into text representations. Specifically, it encodes texts and label annotations independently and then integrates label knowledge into text representation with an elaborate-designed semantics fusion module. We conduct extensive experiments on three typical span extraction tasks: flat NER, nested NER, and event detection. The empirical results show that 1) our method achieves state-of-the-art performance on four benchmarks, and 2) reduces training time and inference time by 76% and 77% on average, respectively, compared with the QA Formalization paradigm. Our code and data are available at https://github.com/Akeepers/LEAR.