# LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model

arXiv: https://arxiv.org/abs/2304.06248 · HTML: https://mlsanity.com/paper/2304.06248/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 95 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 30 of 49)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [Analyzing the Structure of Attention in a Transformer Language Model](https://mlsanity.com/paper/1906.04284/) (2019)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [A Unified Generative Framework for Various NER Subtasks](https://mlsanity.com/paper/2106.01223/) (2021)
- [Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling](https://mlsanity.com/paper/1703.04826/) (2017)
- [Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1911.01616/) (2019)
- [Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks](https://mlsanity.com/paper/1810.09536/) (2018)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders](https://mlsanity.com/paper/2010.03851/) (2020)
- [Effective LSTMs for Target-Dependent Sentiment Classification](https://mlsanity.com/paper/1512.01100/) (2015)
- [Compound Probabilistic Context-Free Grammars for Grammar Induction](https://mlsanity.com/paper/1906.10225/) (2019)
- [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://mlsanity.com/paper/2105.03654/) (2021)
- [TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking](https://mlsanity.com/paper/2010.13415/) (2020)
- [Tree Transformer: Integrating Tree Structures into Self-Attention](https://mlsanity.com/paper/1909.06639/) (2019)
- [Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders](https://mlsanity.com/paper/1904.02142/) (2019)
- [Neural Language Modeling by Jointly Learning Syntax and Lexicon](https://mlsanity.com/paper/1711.02013/) (2017)
- [Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees](https://mlsanity.com/paper/2103.04350/) (2021)
- [Learning to Compose Task-Specific Tree Structures](https://mlsanity.com/paper/1707.02786/) (2017)
- [PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction](https://mlsanity.com/paper/2106.09895/) (2021)

## Built on by (most cited first, 30 of 43)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation](https://mlsanity.com/paper/2308.05095/) (2023)
- [Reasoning Implicit Sentiment with Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.11255/) (2023)
- [Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucination](https://mlsanity.com/paper/2305.12256/) (2023)
- [Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling](https://mlsanity.com/paper/2401.14556/) (2024)
- [Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling](https://mlsanity.com/paper/2305.11719/) (2023)
- [Generating Visual Spatial Description via Holistic 3D Scene Understanding](https://mlsanity.com/paper/2305.11768/) (2023)
- [Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment](https://mlsanity.com/paper/2305.12260/) (2023)
- [Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling](https://mlsanity.com/paper/2308.05081/) (2023)
- [Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition](https://mlsanity.com/paper/2308.04502/) (2023)
- [Mirror: A Universal Framework for Various Information Extraction Tasks](https://mlsanity.com/paper/2311.05419/) (2023)
- [Text2NKG: Fine-Grained N-ary Relation Extraction for N-ary relational Knowledge Graph Construction](https://mlsanity.com/paper/2310.05185/) (2023)
- [ChatEL: Entity Linking with Chatbots](https://mlsanity.com/paper/2402.14858/) (2024)
- [EntGPT: Entity Linking with Generative Large Language Models](https://mlsanity.com/paper/2402.06738/) (2024)
- [Constructing Code-mixed Universal Dependency Forest for Unbiased Cross-lingual Relation Extraction](https://mlsanity.com/paper/2305.12258/) (2023)
- [Informed Named Entity Recognition Decoding for Generative Language Models](https://mlsanity.com/paper/2308.07791/) (2023)
- [Easy-to-Hard Learning for Information Extraction](https://mlsanity.com/paper/2305.09193/) (2023)
- [An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2401.01326/) (2024)
- [Event Extraction in Large Language Model](https://mlsanity.com/paper/2512.19537/) (2025)
- [XNLP: An Interactive Demonstration System for Universal Structured NLP](https://mlsanity.com/paper/2308.01846/) (2023)
- [In-Context Learning for Few-Shot Nested Named Entity Recognition](https://mlsanity.com/paper/2402.01182/) (2024)
- [ECQED: Emotion-Cause Quadruple Extraction in Dialogs](https://mlsanity.com/paper/2306.03969/) (2023)
- [TKDP: Threefold Knowledge-enriched Deep Prompt Tuning for Few-shot Named Entity Recognition](https://mlsanity.com/paper/2306.03974/) (2023)
- [Revisiting Conversation Discourse for Dialogue Disentanglement](https://mlsanity.com/paper/2306.03975/) (2023)
- [A Bi-directional Multi-hop Inference Model for Joint Dialog Sentiment Classification and Act Recognition](https://mlsanity.com/paper/2308.04424/) (2023)
- [Benchingmaking Large Langage Models in Biomedical Triple Extraction](https://mlsanity.com/paper/2310.18463/) (2023)
- [Span-Oriented Information Extraction -- A Unifying Perspective on Information Extraction](https://mlsanity.com/paper/2403.15453/) (2024)
- [GraphER: A Structure-aware Text-to-Graph Model for Entity and Relation Extraction](https://mlsanity.com/paper/2404.12491/) (2024)
- [A Structure-aware Generative Model for Biomedical Event Extraction](https://mlsanity.com/paper/2408.06583/) (2024)
- [PGSO: Prompt-based Generative Sequence Optimization Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2412.00763/) (2024)

## Abstract (arXiv)

Universally modeling all typical information extraction tasks (UIE) with one generative language model (GLM) has revealed great potential by the latest study, where various IE predictions are unified into a linearized hierarchical expression under a GLM. Syntactic structure information, a type of effective feature which has been extensively utilized in IE community, should also be beneficial to UIE. In this work, we propose a novel structure-aware GLM, fully unleashing the power of syntactic knowledge for UIE. A heterogeneous structure inductor is explored to unsupervisedly induce rich heterogeneous structural representations by post-training an existing GLM. In particular, a structural broadcaster is devised to compact various latent trees into explicit high-order forests, helping to guide a better generation during decoding. We finally introduce a task-oriented structure fine-tuning mechanism, further adjusting the learned structures to most coincide with the end-task's need. Over 12 IE benchmarks across 7 tasks our system shows significant improvements over the baseline UIE system. Further in-depth analyses show that our GLM learns rich task-adaptive structural bias that greatly resolves the UIE crux, the long-range dependence issue and boundary identifying. Source codes are open at https://github.com/ChocoWu/LasUIE.