# Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field

arXiv: https://arxiv.org/abs/2212.08929 · HTML: https://mlsanity.com/paper/2212.08929/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 17 Dec 2022.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Conditional Random Fields as Recurrent Neural Networks](https://mlsanity.com/paper/1502.03240/) (2015)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Named Entity Recognition as Dependency Parsing](https://mlsanity.com/paper/2005.07150/) (2020)
- [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://mlsanity.com/paper/2106.09232/) (2021)
- [Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme](https://mlsanity.com/paper/1706.05075/) (2017)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders](https://mlsanity.com/paper/2010.03851/) (2020)
- [Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction](https://mlsanity.com/paper/2202.12109/) (2022)
- [Packed Levitated Marker for Entity and Relation Extraction](https://mlsanity.com/paper/2109.06067/) (2021)
- [A Partition Filter Network for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2108.12202/) (2021)
- [UniRE: A Unified Label Space for Entity Relation Extraction](https://mlsanity.com/paper/2107.04292/) (2021)
- [One for All: Neural Joint Modeling of Entities and Events](https://mlsanity.com/paper/1812.00195/) (2018)
- [Dynamic Prefix-Tuning for Generative Template-based Event Extraction](https://mlsanity.com/paper/2205.06166/) (2022)
- [Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks](https://mlsanity.com/paper/2103.09330/) (2021)
- [Second-Order Semantic Dependency Parsing with End-to-End Neural Networks](https://mlsanity.com/paper/1906.07880/) (2019)
- [Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers](https://mlsanity.com/paper/1902.01030/) (2019)
- [Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training](https://mlsanity.com/paper/2010.05003/) (2020)
- [Integrating Deep Learning with Logic Fusion for Information Extraction](https://mlsanity.com/paper/1912.03041/) (2019)
- [Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing](https://mlsanity.com/paper/2203.04665/) (2022)
- [Regularized Frank-Wolfe for Dense CRFs: Generalizing Mean Field and Beyond](https://mlsanity.com/paper/2110.14759/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Decoupling and Aggregating Framework for Joint Extraction of Entities and Relations](https://mlsanity.com/paper/2405.08311/) (2024)

## Abstract (arXiv)

Prior works on joint Information Extraction (IE) typically model instance (e.g., event triggers, entities, roles, relations) interactions by representation enhancement, type dependencies scoring, or global decoding. We find that the previous models generally consider binary type dependency scoring of a pair of instances, and leverage local search such as beam search to approximate global solutions. To better integrate cross-instance interactions, in this work, we introduce a joint IE framework (CRFIE) that formulates joint IE as a high-order Conditional Random Field. Specifically, we design binary factors and ternary factors to directly model interactions between not only a pair of instances but also triplets. Then, these factors are utilized to jointly predict labels of all instances. To address the intractability problem of exact high-order inference, we incorporate a high-order neural decoder that is unfolded from a mean-field variational inference method, which achieves consistent learning and inference. The experimental results show that our approach achieves consistent improvements on three IE tasks compared with our baseline and prior work.