mlsanityData to 7 Oct 2026

AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters

arXiv 2507.10586 ↗First posted 11 Jul 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable fluency across a range of natural language tasks, yet remain vulnerable to hallucinations - factual inaccuracies that undermine trust in real world deployment. We present AutoRAG-LoRA, a modular framework for Retrieval-Augmented Generation (RAG) that tackles hallucination in large language models through lightweight LoRA-based adapters and KL-regularized training. Our pipeline integrates automated prompt rewriting, hybrid retrieval, and low-rank adapter tuning to ground responses in retrieved evidence. A hallucination detection module, using both classifier-based and self-evaluation techniques, assigns confidence scores to generated outputs, triggering an optional feedback correction loop. This loop enforces factual alignment via contrastive KL loss and adapter fine tuning. We demonstrate that AutoRAG-LoRA significantly reduces the factual drift while preserving the efficiency and modularity of the model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 22 references that are in the map. First posted on arXiv 11 Jul 2025.

Builds on

9 of 22 references.

References · 9

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  9. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs
  2. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
  3. 2025LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals
  4. 2024THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation in Large Language Models
  5. 2025REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models
  6. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
  7. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
  8. 2024Lynx: An Open Source Hallucination Evaluation Model