# When to Use What: An In-Depth Comparative Empirical Analysis of OpenIE Systems for Downstream Applications

arXiv: https://arxiv.org/abs/2211.08228 · HTML: https://mlsanity.com/paper/2211.08228/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 17 of 17)

- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Faithful to the Original: Fact Aware Neural Abstractive Summarization](https://mlsanity.com/paper/1711.04434/) (2017)
- [A Survey on Open Information Extraction](https://mlsanity.com/paper/1806.05599/) (2018)
- [Neural Open Information Extraction](https://mlsanity.com/paper/1805.04270/) (2018)
- [IMoJIE: Iterative Memory-Based Joint Open Information Extraction](https://mlsanity.com/paper/2005.08178/) (2020)
- [Span Model for Open Information Extraction on Accurate Corpus](https://mlsanity.com/paper/1901.10879/) (2019)
- [A Survey on Neural Open Information Extraction: Current Status and Future Directions](https://mlsanity.com/paper/2205.11725/) (2022)
- [Multi2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT](https://mlsanity.com/paper/2009.08128/) (2020)
- [Answering Complex Questions Using Open Information Extraction](https://mlsanity.com/paper/1704.05572/) (2017)
- [BLEU, METEOR, BERTScore: Evaluation of Metrics Performance in Assessing Critical Translation Errors in Sentiment-oriented Text](https://mlsanity.com/paper/2109.14250/) (2021)
- [Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs](https://mlsanity.com/paper/1908.00469/) (2019)
- [WiRe57 : A Fine-Grained Benchmark for Open Information Extraction](https://mlsanity.com/paper/1809.08962/) (2018)
- [LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction](https://mlsanity.com/paper/2101.11177/) (2021)
- [DetIE: Multilingual Open Information Extraction Inspired by Object Detection](https://mlsanity.com/paper/2206.12514/) (2022)
- [Supervising Unsupervised Learning](https://mlsanity.com/paper/1709.05262/) (2017)
- [Complex Question Answering: Unsupervised Learning Approaches and Experiments](https://mlsanity.com/paper/1401.3479/) (2014)

## Built on by (most cited first, 4 of 4)

- [HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models](https://mlsanity.com/paper/2405.14831/) (2024)
- [Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction](https://mlsanity.com/paper/2305.13981/) (2023)
- [Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment](https://mlsanity.com/paper/2310.10590/) (2023)
- [textitBenchIEFL : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark](https://mlsanity.com/paper/2407.16860/) (2024)

## Abstract (arXiv)

Open Information Extraction (OpenIE) has been used in the pipelines of various NLP tasks. Unfortunately, there is no clear consensus on which models to use in which tasks. Muddying things further is the lack of comparisons that take differing training sets into account. In this paper, we present an application-focused empirical survey of neural OpenIE models, training sets, and benchmarks in an effort to help users choose the most suitable OpenIE systems for their applications. We find that the different assumptions made by different models and datasets have a statistically significant effect on performance, making it important to choose the most appropriate model for one's applications. We demonstrate the applicability of our recommendations on a downstream Complex QA application.