mlsanityData to 7 Oct 2026

TruthTorchLM: A Comprehensive Library for Predicting Truthfulness in LLM Outputs

arXiv 2507.08203 ↗First posted 10 Jul 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Generative Large Language Models (LLMs)inevitably produce untruthful responses. Accurately predicting the truthfulness of these outputs is critical, especially in high-stakes settings. To accelerate research in this domain and make truthfulness prediction methods more accessible, we introduce TruthTorchLM an open-source, comprehensive Python library featuring over 30 truthfulness prediction methods, which we refer to as Truth Methods. Unlike existing toolkits such as Guardrails, which focus solely on document-grounded verification, or LM-Polygraph, which is limited to uncertainty-based methods, TruthTorchLM offers a broad and extensible collection of techniques. These methods span diverse tradeoffs in computational cost, access level (e.g., black-box vs white-box), grounding document requirements, and supervision type (self-supervised or supervised). TruthTorchLM is seamlessly compatible with both HuggingFace and LiteLLM, enabling support for locally hosted and API-based models. It also provides a unified interface for generation, evaluation, calibration, and long-form truthfulness prediction, along with a flexible framework for extending the library with new methods. We conduct an evaluation of representative truth methods on three datasets, TriviaQA, GSM8K, and FactScore-Bio. The code is available at https://github.com/Ybakman/TruthTorchLM

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

22 of 34 references.

References · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  4. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  5. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  6. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  7. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  8. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  9. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  10. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  11. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  12. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
10 more
  1. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 130
  2. 2024LUQ: Long-text Uncertainty Quantification for LLMscited by 79
  3. 2024Lynx: An Open Source Hallucination Evaluation Modelcited by 55
  4. 2023Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Methodcited by 51
  5. 2022Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Taskscited by 49
  6. 2024MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMscited by 45
  7. 2023Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinationscited by 34
  8. 2024Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMscited by 23
  9. 2024LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentationcited by 22
  10. 2025Reconsidering LLM Uncertainty Estimation Methods in the Wildcited by 17

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Uncertainty Quantification for Hallucination Detection in Large Language Models: Foundations, Methodology, and Future Directionscited by 10
  2. 2025Challenges for Generative AI in Legal Reasoningcited by 7
  3. 2025Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question-Answeringcited by 6
  4. 2025Uncertainty Quantification for Retrieval-Augmented Reasoningcited by 5