# Multi2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT

arXiv: https://arxiv.org/abs/2009.08128 · HTML: https://mlsanity.com/paper/2009.08128/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 26 of 26)

- [A Primer on Pretrained Multilingual Language Models](https://mlsanity.com/paper/2107.00676/) (2021)
- [A Survey on Neural Open Information Extraction: Current Status and Future Directions](https://mlsanity.com/paper/2205.11725/) (2022)
- [Zero-Shot Information Extraction as a Unified Text-to-Triple Translation](https://mlsanity.com/paper/2109.11171/) (2021)
- [BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation](https://mlsanity.com/paper/2109.06850/) (2021)
- [DetIE: Multilingual Open Information Extraction Inspired by Object Detection](https://mlsanity.com/paper/2206.12514/) (2022)
- [A Survey on Open Information Extraction from Rule-based Model to Large Language Model](https://mlsanity.com/paper/2208.08690/) (2022)
- [Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction](https://mlsanity.com/paper/2305.13981/) (2023)
- [AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark](https://mlsanity.com/paper/2109.07464/) (2021)
- [mOKB6: A Multilingual Open Knowledge Base Completion Benchmark](https://mlsanity.com/paper/2211.06959/) (2022)
- [Glitter or Gold? Deriving Structured Insights from Sustainability Reports via Large Language Models](https://mlsanity.com/paper/2310.05628/) (2023)
- [CompactIE: Compact Facts in Open Information Extraction](https://mlsanity.com/paper/2205.02880/) (2022)
- [Linking Surface Facts to Large-Scale Knowledge Graphs](https://mlsanity.com/paper/2310.14909/) (2023)
- [XFormParser: A Simple and Effective Multimodal Multilingual Semi-structured Form Parser](https://mlsanity.com/paper/2405.17336/) (2024)
- [When to Use What: An In-Depth Comparative Empirical Analysis of OpenIE Systems for Downstream Applications](https://mlsanity.com/paper/2211.08228/) (2022)
- [PENELOPIE: Enabling Open Information Extraction for the Greek Language through Machine Translation](https://mlsanity.com/paper/2103.15075/) (2021)
- [milIE: Modular & Iterative Multilingual Open Information Extraction](https://mlsanity.com/paper/2110.08144/) (2021)
- [State of the Art in Artificial Intelligence applied to the Legal Domain](https://mlsanity.com/paper/2204.07047/) (2022)
- [MT4CrossOIE: Multi-stage Tuning for Cross-lingual Open Information Extraction](https://mlsanity.com/paper/2308.06552/) (2023)
- [IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models](https://mlsanity.com/paper/2210.14128/) (2022)
- [Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment](https://mlsanity.com/paper/2310.10590/) (2023)
- [Zero-Shot Open-Schema Entity Structure Discovery](https://mlsanity.com/paper/2506.04458/) (2025)
- [NL2GDPR: Automatically Develop GDPR Compliant Android Application Features from Natural Language](https://mlsanity.com/paper/2208.13361/) (2022)
- [Distantly Supervised Morpho-Syntactic Model for Relation Extraction](https://mlsanity.com/paper/2401.10002/) (2024)
- [HYBRINFOX at CheckThat! 2024 -- Task 1: Enhancing Language Models with Structured Information for Check-Worthiness Estimation](https://mlsanity.com/paper/2407.03850/) (2024)
- [textitBenchIEFL : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark](https://mlsanity.com/paper/2407.16860/) (2024)
- [Extracting Structured Requirements from Unstructured Building Technical Specifications for Building Information Modeling](https://mlsanity.com/paper/2508.13833/) (2025)

## Abstract (arXiv)

In this paper, we propose Multi2OIE, which performs open information extraction (open IE) by combining BERT with multi-head attention. Our model is a sequence-labeling system with an efficient and effective argument extraction method. We use a query, key, and value setting inspired by the Multimodal Transformer to replace the previously used bidirectional long short-term memory architecture with multi-head attention. Multi2OIE outperforms existing sequence-labeling systems with high computational efficiency on two benchmark evaluation datasets, Re-OIE2016 and CaRB. Additionally, we apply the proposed method to multilingual open IE using multilingual BERT. Experimental results on new benchmark datasets introduced for two languages (Spanish and Portuguese) demonstrate that our model outperforms other multilingual systems without training data for the target languages.