# LLM-OREF: An Open Relation Extraction Framework Based on Large Language Models

arXiv: https://arxiv.org/abs/2509.15089 · HTML: https://mlsanity.com/paper/2509.15089/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Macro F1 and Macro F1](https://mlsanity.com/paper/1911.03347/) (2019)
- [Improved Neural Relation Detection for Knowledge Base Question Answering](https://mlsanity.com/paper/1704.06194/) (2017)
- [SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction](https://mlsanity.com/paper/2004.02438/) (2020)
- [A Label Dependence-aware Sequence Generation Model for Multi-level Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2112.11740/) (2021)
- [A Relation-Oriented Clustering Method for Open Relation Extraction](https://mlsanity.com/paper/2109.07205/) (2021)
- [Unsupervised Open Relation Extraction](https://mlsanity.com/paper/1801.07174/) (2018)
- [Leveraging Dependency Forest for Neural Medical Relation Extraction](https://mlsanity.com/paper/1911.04123/) (2019)
- [Improving Unsupervised Relation Extraction by Augmenting Diverse Sentence Pairs](https://mlsanity.com/paper/2312.00552/) (2023)
- [Towards Better Document-level Relation Extraction via Iterative Inference](https://mlsanity.com/paper/2211.14470/) (2022)
- [Towards Better Graph-based Cross-document Relation Extraction via Non-bridge Entity Enhancement and Prediction Debiasing](https://mlsanity.com/paper/2406.16529/) (2024)
- [Towards a More Generalized Approach in Open Relation Extraction](https://mlsanity.com/paper/2505.22801/) (2025)

## Abstract (arXiv)

The goal of open relation extraction (OpenRE) is to develop an RE model that can generalize to new relations not encountered during training. Existing studies primarily formulate OpenRE as a clustering task. They first cluster all test instances based on the similarity between the instances, and then manually assign a new relation to each cluster. However, their reliance on human annotation limits their practicality. In this paper, we propose an OpenRE framework based on large language models (LLMs), which directly predicts new relations for test instances by leveraging their strong language understanding and generation abilities, without human intervention. Specifically, our framework consists of two core components: (1) a relation discoverer (RD), designed to predict new relations for test instances based on textitdemonstrations formed by training instances with known relations; and (2) a relation predictor (RP), used to select the most likely relation for a test instance from n candidate relations, guided by textitdemonstrations composed of their instances. To enhance the ability of our framework to predict new relations, we design a self-correcting inference strategy composed of three stages: relation discovery, relation denoising, and relation prediction. In the first stage, we use RD to preliminarily predict new relations for all test instances. Next, we apply RP to select some high-reliability test instances for each new relation from the prediction results of RD through a cross-validation method. During the third stage, we employ RP to re-predict the relations of all test instances based on the demonstrations constructed from these reliable test instances. Extensive experiments on three OpenRE datasets demonstrate the effectiveness of our framework. We release our code at https://github.com/XMUDeepLIT/LLM-OREF.git.