mlsanityData to 7 Oct 2026

Core: Robust Factual Precision with Informative Sub-Claim Identification

arXiv 2407.03572 ↗First posted 4 Jul 2024cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Hallucinations pose a challenge to the application of large language models (LLMs) thereby motivating the development of metrics to evaluate factual precision. We observe that popular metrics using the Decompose-Then-Verify framework, such as FActScore, can be manipulated by adding obvious or repetitive subclaims to artificially inflate scores. This observation motivates our new customizable plug-and-play subclaim selection component called Core, which filters down individual subclaims according to their uniqueness and informativeness. We show that many popular factual precision metrics augmented by Core are substantially more robust on a wide range of knowledge domains. We release an evaluation framework supporting easy and modular use of Core and various decomposition strategies, which we recommend adoption by the community. We also release an expansion of the FActScore biography dataset to facilitate further studies of decomposition-based factual precision evaluation.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

34 of 53 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  11. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
22 more
  1. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  2. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  3. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  4. 2021A Survey on Automated Fact-Checkingcited by 376
  5. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  6. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  7. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  8. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  9. 2023Fine-tuning Language Models for Factualitycited by 219
  10. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  11. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  12. 2024Long-form factuality in large language modelscited by 146
  13. 2024Language Models with Conformal Factuality Guaranteescited by 128
  14. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  15. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  16. 2021Decontextualization: Making Sentences Stand-Alonecited by 101
  17. 2023LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarizationcited by 96
  18. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 96
  19. 2024A Closer Look at Claim Decompositioncited by 40
  20. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31
  21. 2024Towards a Holistic Evaluation of LLMs on Factual Knowledge Recallcited by 8
  22. 2023Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Stylescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 6.

15 papers.

Sort:

By area:

  1. 2024DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generationcited by 22
  2. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
  3. 2025Optimizing Decomposition for Optimal Claim Verificationcited by 10
  4. 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimationcited by 9
  5. 2025All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generationscited by 6
  6. 2025BOOST: Bootstrapping Strategy-Driven Reasoning Programs for Program-Guided Fact-Checkingcited by 3
  7. 2025Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generationcited by 1
  8. 2025QA-Noun: Representing Nominal Semantics via Natural Language Question-Answer Pairscited by 1
  9. 2025How Grounded is Wikipedia? A Study on Structured Evidential Support and Retrievalcited by 0
  10. 2026Robust Uncertainty Quantification for Factual Generation of Large Language Modelscited by 0
  11. 2026VetScore: Risk-Weighted Fact Verification for Veterinary Long-Form QA with Citationscited by 0
  12. 2026Information Satisfaction: A Reader-Centered Axis for Summarization Evaluationcited by 0
3 more
  1. 2026ElementCheck: Complexity-Aware Long-Form Text Factuality Evaluation via Sentence Elementscited by 0
  2. 2026The Privacy-Hallucination Tradeoff in Differentially Private Language Modelscited by 0
  3. 2026FrankenReport: Early Exiting in Long-Form Generation Using Expected Value of Computationcited by 0