# KnowsLM: A framework for evaluation of small language models for knowledge augmentation and humanised conversations

arXiv: https://arxiv.org/abs/2504.04569 · HTML: https://mlsanity.com/paper/2504.04569/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 20 references that are in the map.
- First posted on arXiv 6 Apr 2025.

## Builds on (most cited first, 17 of 17)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness](https://mlsanity.com/paper/2411.03350/) (2024)
- [Evaluating the Efficacy of Open-Source LLMs in Enterprise-Specific RAG Systems: A Comparative Study of Performance and Scalability](https://mlsanity.com/paper/2406.11424/) (2024)
- [Keyword Augmented Retrieval: Novel framework for Information Retrieval integrated with speech interface](https://mlsanity.com/paper/2310.04205/) (2023)
- [KnowTuning: Knowledge-aware Fine-tuning for Large Language Models](https://mlsanity.com/paper/2402.11176/) (2024)
- [Introducing a new hyper-parameter for RAG: Context Window Utilization](https://mlsanity.com/paper/2407.19794/) (2024)
- [GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning](https://mlsanity.com/paper/2412.09250/) (2024)
- [Leveraging LLMs for Dialogue Quality Measurement](https://mlsanity.com/paper/2406.17304/) (2024)
- [COS-Mix: Cosine Similarity and Distance Fusion for Improved Information Retrieval](https://mlsanity.com/paper/2406.00638/) (2024)
- [CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models](https://mlsanity.com/paper/2409.02119/) (2024)
- [VidyaRANG: Conversational Learning Based Platform powered by Large Language Model](https://mlsanity.com/paper/2407.16209/) (2024)
- [CultureVo: The Serious Game of Utilizing Gen AI for Enhancing Cultural Intelligence](https://mlsanity.com/paper/2407.20685/) (2024)

## Built on by (most cited first, 6 of 6)

- [A Survey on Collaborative Mechanisms Between Large and Small Language Models](https://mlsanity.com/paper/2505.07460/) (2025)
- [AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMs](https://mlsanity.com/paper/2507.18584/) (2025)
- [VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality](https://mlsanity.com/paper/2509.06994/) (2025)
- [Prescriptive Agents based on RAG for Automated Maintenance (PARAM)](https://mlsanity.com/paper/2508.04714/) (2025)
- [E-ARMOR: Edge case Assessment and Review of Multilingual Optical Character Recognition](https://mlsanity.com/paper/2509.03615/) (2025)
- [M-PACE: Mother Child Framework for Multimodal Compliance](https://mlsanity.com/paper/2509.15241/) (2025)

## Abstract (arXiv)

In the evolving landscape of conversational AI, generating concise, context-aware, and human-like dialogue using small and medium-sized language models (LLMs) remains a complex challenge. This study investigates the influence of LoRA rank, dataset scale, and prompt prefix design on both knowledge retention and stylistic alignment. While fine-tuning improves fluency and enables stylistic customization, its ability to integrate unseen knowledge is constrained -- particularly with smaller datasets. Conversely, RAG-augmented models, equipped to incorporate external documents at inference, demonstrated superior factual accuracy on out-of-distribution prompts, though they lacked the stylistic consistency achieved by fine-tuning. Evaluations by LLM-based judges across knowledge accuracy, conversational quality, and conciseness suggest that fine-tuning is best suited for tone adaptation, whereas RAG excels at real-time knowledge augmentation.