# Fine-tuning and Prompt Engineering with Cognitive Knowledge Graphs for Scholarly Knowledge Organization

arXiv: https://arxiv.org/abs/2409.06433 · HTML: https://mlsanity.com/paper/2409.06433/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 10 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2304.01933/) (2023)
- [QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2309.14717/) (2023)
- [Style Over Substance: Evaluation Biases for Large Language Models](https://mlsanity.com/paper/2307.03025/) (2023)
- [LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding](https://mlsanity.com/paper/2306.14924/) (2023)
- [Complex Logical Reasoning over Knowledge Graphs using Large Language Models](https://mlsanity.com/paper/2305.01157/) (2023)
- [Limits for Learning with Language Models](https://mlsanity.com/paper/2306.12213/) (2023)
- [Prefix Propagation: Parameter-Efficient Tuning for Long Sequences](https://mlsanity.com/paper/2305.12086/) (2023)
- [Gemini Pro Defeated by GPT-4V: Evidence from Education](https://mlsanity.com/paper/2401.08660/) (2023)
- [Evaluating Prompt-based Question Answering for Object Prediction in the Open Research Knowledge Graph](https://mlsanity.com/paper/2305.12900/) (2023)

## Built on by (most cited first, 2 of 2)

- [Iterative Hypothesis Generation for Scientific Discovery with Monte Carlo Nash Equilibrium Self-Refining Trees](https://mlsanity.com/paper/2503.19309/) (2025)
- [On the Effectiveness of Large Language Models in Automating Categorization of Scientific Texts](https://mlsanity.com/paper/2502.15745/) (2025)

## Abstract (arXiv)

The increasing amount of published scholarly articles, exceeding 2.5 million yearly, raises the challenge for researchers in following scientific progress. Integrating the contributions from scholarly articles into a novel type of cognitive knowledge graph (CKG) will be a crucial element for accessing and organizing scholarly knowledge, surpassing the insights provided by titles and abstracts. This research focuses on effectively conveying structured scholarly knowledge by utilizing large language models (LLMs) to categorize scholarly articles and describe their contributions in a structured and comparable manner. While previous studies explored language models within specific research domains, the extensive domain-independent knowledge captured by LLMs offers a substantial opportunity for generating structured contribution descriptions as CKGs. Additionally, LLMs offer customizable pathways through prompt engineering or fine-tuning, thus facilitating to leveraging of smaller LLMs known for their efficiency, cost-effectiveness, and environmental considerations. Our methodology involves harnessing LLM knowledge, and complementing it with domain expert-verified scholarly data sourced from a CKG. This strategic fusion significantly enhances LLM performance, especially in tasks like scholarly article categorization and predicate recommendation. Our method involves fine-tuning LLMs with CKG knowledge and additionally injecting knowledge from a CKG with a novel prompting technique significantly increasing the accuracy of scholarly knowledge extraction. We integrated our approach in the Open Research Knowledge Graph (ORKG), thus enabling precise access to organized scholarly knowledge, crucially benefiting domain-independent scholarly knowledge exchange and dissemination among policymakers, industrial practitioners, and the general public.