# KSOD: Knowledge Supplement for LLMs On Demand

arXiv: https://arxiv.org/abs/2503.07550 · HTML: https://mlsanity.com/paper/2503.07550/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [LoRA Learns Less and Forgets Less](https://mlsanity.com/paper/2405.09673/) (2024)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [Understanding Catastrophic Forgetting in Language Models via Implicit Inference](https://mlsanity.com/paper/2309.10105/) (2023)
- [Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails](https://mlsanity.com/paper/2501.09004/) (2025)
- [Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement](https://mlsanity.com/paper/2402.11436/) (2024)
- [Learning From Mistakes Makes LLM Better Reasoner](https://mlsanity.com/paper/2310.20689/) (2023)
- [Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning](https://mlsanity.com/paper/2403.20046/) (2024)
- [DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion](https://mlsanity.com/paper/1902.10526/) (2019)
- [Prompting open-source and commercial language models for grammatical error correction of English learner text](https://mlsanity.com/paper/2401.07702/) (2024)
- [Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Data](https://mlsanity.com/paper/2312.12832/) (2023)
- [Enhancing Grammatical Error Correction Systems with Explanations](https://mlsanity.com/paper/2305.15676/) (2023)
- [Sparsely Shared LoRA on Whisper for Child Speech Recognition](https://mlsanity.com/paper/2309.11756/) (2023)
- [LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement](https://mlsanity.com/paper/2407.00497/) (2024)
- [Implicit Discourse Relation Identification for Open-domain Dialogues](https://mlsanity.com/paper/1907.03975/) (2019)
- [Semantically Driven Sentence Fusion: Modeling and Evaluation](https://mlsanity.com/paper/2010.02592/) (2020)
- [Unveiling and Addressing Pseudo Forgetting in Large Language Models](https://mlsanity.com/paper/2411.11932/) (2024)
- [WikiSplit++: Easy Data Refinement for Split and Rephrase](https://mlsanity.com/paper/2404.09002/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, yet still produce errors in domain-specific tasks. To further improve their performance, we propose KSOD (Knowledge Supplement for LLMs On Demand), a novel framework that empowers LLMs to improve their capabilities with knowledge-based supervised fine-tuning (SFT). KSOD analyzes the causes of errors from the perspective of knowledge deficiency by identifying potential missing knowledge in LLM that may lead to the errors. Subsequently, KSOD tunes a knowledge module on knowledge dataset and verifies whether the LLM lacks the identified knowledge based on it. If the knowledge is verified, KSOD supplements the LLM with the identified knowledge using the knowledge module. Tuning LLMs on specific knowledge instead of specific task decouples task and knowledge and our experiments on two domain-specific benchmarks and four general benchmarks empirically demonstrate that KSOD enhances the performance of LLMs on tasks requiring the supplemented knowledge while preserving their performance on other tasks. Our findings shed light on the potential of improving the capabilities of LLMs with knowledge-based SFT.