mlsanityData to 14 Oct 2026

FaaF: Facts as a Function for the evaluation of generated text

arXiv 2403.03888 ↗First posted 6 Mar 2024cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

The demand for accurate and efficient verification of information in texts generated by large language models (LMs) is at an all-time high, but remains unresolved. Recent efforts have focused on extracting and verifying atomic facts from these texts via prompting LM evaluators. However, we demonstrate that this method of prompting is unreliable when faced with incomplete or inaccurate reference information. We introduce Facts as a Function (FaaF), a new approach to the fact verification task that leverages the function-calling capabilities of LMs. FaaF significantly enhances the ability of LMs to identify unsupported facts in texts, while also improving efficiency and significantly lowering costs compared to prompt-based methods. Additionally, we propose a framework for evaluating factual recall in Retrieval Augmented Generation (RAG) systems, which we employ to compare prompt-based and FaaF methods using various LMs under challenging conditions.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 6 Mar 2024.

Builds on

14 of 24 references.

References · 14

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  5. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
  6. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  7. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  8. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 418
  9. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
  10. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  11. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  12. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
2 more
  1. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 167
  2. 2022Generating Literal and Implied Subquestions to Fact-check Complex Claimscited by 66

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 2.

5 papers.

  1. 2024Evaluating RAG-Fusion with RAGElo: an Automated Elo-based Frameworkcited by 11
  2. 2024Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systemscited by 8
  3. 2025Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasetscited by 7
  4. 2024A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systemscited by 3
  5. 2025Automatic Fact-checking in English and Telugucited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024UFO: a Unified and Flexible Framework for Evaluating Factuality of Large Language Models
  2. 2024Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking
  3. 2024Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown
  4. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
  5. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
  6. 2025Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs
  7. 2024FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation
  8. 2024Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output