# HySPA: Hybrid Span Generation for Scalable Text-to-Graph Extraction

arXiv: https://arxiv.org/abs/2106.15838 · HTML: https://mlsanity.com/paper/2106.15838/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [Sequence-to-Sequence Learning as Beam-Search Optimization](https://mlsanity.com/paper/1606.02960/) (2016)
- [A Survey of Heterogeneous Information Network Analysis](https://mlsanity.com/paper/1511.04854/) (2015)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders](https://mlsanity.com/paper/2010.03851/) (2020)
- [CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning](https://mlsanity.com/paper/1911.10438/) (2019)
- [IMoJIE: Iterative Memory-Based Joint Open Information Extraction](https://mlsanity.com/paper/2005.08178/) (2020)
- [Broad-Coverage Semantic Parsing as Transduction](https://mlsanity.com/paper/1909.02607/) (2019)
- [Minimize Exposure Bias of Seq2Seq Models in Joint Entity and Relation Extraction](https://mlsanity.com/paper/2009.07503/) (2020)
- [Seq2RDF: An end-to-end application for deriving Triples from Natural Language Text](https://mlsanity.com/paper/1807.01763/) (2018)

## Built on by (most cited first, 6 of 6)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2401.01326/) (2024)
- [Span-based joint entity and relation extraction augmented with sequence tagging mechanism](https://mlsanity.com/paper/2210.12720/) (2022)
- [PyTorch-IE: Fast and Reproducible Prototyping for Information Extraction](https://mlsanity.com/paper/2406.00007/) (2024)

## Abstract (arXiv)

Text-to-Graph extraction aims to automatically extract information graphs consisting of mentions and types from natural language texts. Existing approaches, such as table filling and pairwise scoring, have shown impressive performance on various information extraction tasks, but they are difficult to scale to datasets with longer input texts because of their second-order space/time complexities with respect to the input length. In this work, we propose a Hybrid Span Generator (HySPA) that invertibly maps the information graph to an alternating sequence of nodes and edge types, and directly generates such sequences via a hybrid span decoder which can decode both the spans and the types recurrently in linear time and space complexities. Extensive experiments on the ACE05 dataset show that our approach also significantly outperforms state-of-the-art on the joint entity and relation extraction task.