mlsanityData to 7 Oct 2026

FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios

arXiv 2307.13528 ↗First posted 25 Jul 2023cs.CL cs.AIcited by 247

Not on the map. Why?

Cited by 25 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The emergence of generative pre-trained models has facilitated the synthesis of high-quality text, but it has also posed challenges in identifying factual errors in the generated text. In particular: (1) A wider range of tasks now face an increasing risk of containing factual errors when handled by generative models. (2) Generated texts tend to be lengthy and lack a clearly defined granularity for individual facts. (3) There is a scarcity of explicit evidence available during the process of fact checking. With the above challenges in mind, in this paper, we propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models (e.g., ChatGPT). Experiments on four different tasks (knowledge-based QA, code generation, mathematical reasoning, and scientific literature review) show the efficacy of the proposed method. We release the code of FacTool associated with ChatGPT plugin interface at https://github.com/GAIR-NLP/factool .

Key facts. Cited by 247 papers in the map (data through 7 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

33 of 48 references.

References · 33

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  12. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
21 more
  1. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  2. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  3. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  4. 2023LIMA: Less Is More for Alignmentcited by 1,080
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2023Teaching Large Language Models to Self-Debugcited by 963
  7. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  8. 2022Galactica: A Large Language Model for Sciencecited by 736
  9. 2022PAL: Program-aided Language Modelscited by 635
  10. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  11. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  12. 2022CodeT: Code Generation with Generated Testscited by 369
  13. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  14. 2021Internet-Augmented Dialogue Generationcited by 217
  15. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  16. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 141
  17. 2022Natural Language to Code Translation with Executioncited by 122
  18. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  19. 2019Reasoning Over Semantic-Level Graph for Fact Checkingcited by 69
  20. 2021ProoFVer: Natural Logic Theorem Proving for Fact Verificationcited by 52
  21. 2021ExplainaBoard: An Explainable Leaderboard for NLPcited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 31; 2024: 106; 2025: 70; 2026: 40.

247 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  4. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  5. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  6. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  7. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  8. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  9. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  10. 2024Tool Learning with Large Language Models: A Surveycited by 221
  11. 2023Fine-tuning Language Models for Factualitycited by 219
  12. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
38 more
  1. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  2. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  3. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  4. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  5. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  6. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  7. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  8. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  9. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  10. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  11. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  12. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 98
  13. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  14. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  15. 2024ContextCite: Attributing Model Generation to Contextcited by 93
  16. 2024O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?cited by 90
  17. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  18. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  19. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  20. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  21. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 84
  22. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  23. 2024On Verbalized Confidence Scores for LLMscited by 83
  24. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 78
  25. 2024Unified Hallucination Detection for Multimodal Large Language Modelscited by 75
  26. 2024Safeguarding Large Language Models: A Surveycited by 73
  27. 2023Alignment for Honestycited by 72
  28. 2023A Survey of Large Language Models Attributioncited by 71
  29. 2023Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofittingcited by 70
  30. 2024Factuality of Large Language Models: A Surveycited by 64
  31. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 63
  32. 2024Building Guardrails for Large Language Modelscited by 61
  33. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 56
  34. 2023LLatrieval: LLM-Verified Retrieval for Verifiable Generationcited by 52
  35. 2024Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden Statescited by 50
  36. 2024Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Modelscited by 47
  37. 2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 46
  38. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoningcited by 41

50 most cited of 247 All 247 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown
  2. 2025Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics
  3. 2025FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data
  4. 2024AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators
  5. 2024Identifying the Achilles' Heel: An Iterative Method for Dynamically Uncovering Factual Errors in Large Language Models
  6. 2025FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models
  7. 2025Entailed Opinion Matters: Improving the Fact-Checking Performance of Language Models by Relying on their Entailment Ability
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation