mlsanityData to 14 Oct 2026

RAGulator: Lightweight Out-of-Context Detectors for Grounded Text Generation

arXiv 2411.03920 ↗First posted 6 Nov 2024cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Real-time detection of out-of-context LLM outputs is crucial for enterprises looking to safely adopt RAG applications. In this work, we train lightweight models to discriminate LLM-generated text that is semantically out-of-context from retrieved text documents. We preprocess a combination of summarisation and semantic textual similarity datasets to construct training data using minimal resources. We find that DeBERTa is not only the best-performing model under this pipeline, but it is also fast and does not require additional text preprocessing or feature engineering. While emerging work demonstrates that generative LLMs can also be fine-tuned and used in complex data pipelines to achieve state-of-the-art performance, we note that speed and resource limits are important considerations for on-premise deployment.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

29 of 41 references.

References · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  8. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  9. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  10. 2023Active Retrieval Augmented Generationcited by 516
  11. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
  12. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
17 more
  1. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  2. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 341
  3. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  4. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 288
  5. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  6. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 265
  7. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  8. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  9. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 149
  10. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 142
  11. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  12. 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunitiescited by 64
  13. 2024Lynx: An Open Source Hallucination Evaluation Modelcited by 57
  14. 2024Discovering Preference Optimization Algorithms with and for Large Language Modelscited by 42
  15. 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
  16. 2024Drowzee: Metamorphic Testing for Fact-Conflicting Hallucination Detection in Large Language Modelscited by 14
  17. 2024Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Costcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agentscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
  2. 2024SFR-RAG: Towards Contextually Faithful LLMs
  3. 2024Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report
  4. 2025Parametric Retrieval Augmented Generation
  5. 2025Analise Semantica Automatizada com LLM e RAG para Bulas Farmaceuticas
  6. 2025Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
  7. 2025Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation
  8. 2025Understanding Parametric Knowledge Injection in Retrieval-Augmented Generation