# textitBenchIEFL : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark

arXiv: https://arxiv.org/abs/2407.16860 · HTML: https://mlsanity.com/paper/2407.16860/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 9 of 9)

- [Span Model for Open Information Extraction on Accurate Corpus](https://mlsanity.com/paper/1901.10879/) (2019)
- [Multi2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT](https://mlsanity.com/paper/2009.08128/) (2020)
- [Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs](https://mlsanity.com/paper/1908.00469/) (2019)
- [WiRe57 : A Fine-Grained Benchmark for Open Information Extraction](https://mlsanity.com/paper/1809.08962/) (2018)
- [LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction](https://mlsanity.com/paper/2101.11177/) (2021)
- [BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation](https://mlsanity.com/paper/2109.06850/) (2021)
- [Assertion-based QA with Question-Aware Open Information Extraction](https://mlsanity.com/paper/1801.07414/) (2018)
- [AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark](https://mlsanity.com/paper/2109.07464/) (2021)
- [When to Use What: An In-Depth Comparative Empirical Analysis of OpenIE Systems for Downstream Applications](https://mlsanity.com/paper/2211.08228/) (2022)

## Abstract (arXiv)

Open Information Extraction (OIE) is a field of natural language processing that aims to present textual information in a format that allows it to be organized, analyzed and reflected upon. Numerous OIE systems are developed, claiming ever-increasing performance, marking the need for objective benchmarks. BenchIE is the latest reference we know of. Despite being very well thought out, we noticed a number of issues we believe are limiting. Therefore, we propose textitBenchIEFL, a new OIE benchmark which fully enforces the principles of BenchIE while containing fewer errors, omissions and shortcomings when candidate facts are matched towards reference ones. textitBenchIEFL allows insightful conclusions to be drawn on the actual performance of OIE extractors.