Ontology-Guided Multi-Agent Extraction of Evaluation Objects from Academic Review Texts: Evidence from Chinese Library and Information Science
Not on the map. Why?
Abstract (arXiv)
Academic reviews, scholarly commentaries, and book reviews serve as sources of evaluative statements about theories, methods, literature, institutions, and policies, providing valuable evidence for scholarly evaluation. Existing scientific entity extraction methods mainly target research articles and are less effective for evaluation objects, which are often abstract, context-dependent, and characterized by ambiguous type boundaries. This study proposes an ontology-guided multi-agent framework for evaluation object extraction. The framework combines candidate discovery, ontology-constrained classification, and domain review. Experimental results show that it achieves a Precision of 90.33%, Recall of 84.55%, Entity-level F1 of 87.34%, Strict Typed F1 of 79.78%, and Type Accuracy of 91.35%, substantially outperforming rule-based and zero-shot baselines. Ablation results indicate that the multi-agent workflow improves recall and stability, while ontology-based boundary constraints enhance fine-grained classification and reduce category confusion. The framework supports the structured utilization of evaluative scholarly texts and provides methodological support for evidence-based research evaluation and STI mining.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 44 references that are in the map. First posted on arXiv 30 Aug 2026.
Builds on
11 of 44 references.
References · 11
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
- 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 177
- 2020SciREX: A Challenge Dataset for Document-Level Information Extractioncited by 113
- 2020CrossNER: Evaluating Cross-Domain Named Entity Recognitioncited by 104
- 2015Co-word Maps and Topic Modeling: A Comparison Using Small and Medium-Sized Corpora (n < 1000)cited by 5
- 2019Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000 post-publication open peer review publishing modelcited by 3
- 2016Professional and Citizen Bibliometrics: Complementarities and ambivalences in the development and use of indicatorscited by 1
Built on by
Not cited yet.