# Learning to Generate and Evaluate Fact-checking Explanations with Transformers

arXiv: https://arxiv.org/abs/2410.15669 · HTML: https://mlsanity.com/paper/2410.15669/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Builds on (most cited first, 30 of 34)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Towards a Unified Multi-Dimensional Evaluator for Text Generation](https://mlsanity.com/paper/2210.07197/) (2022)
- [Explainable Automated Fact-Checking for Public Health Claims](https://mlsanity.com/paper/2010.09926/) (2020)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Generating Fact Checking Explanations](https://mlsanity.com/paper/2004.05773/) (2020)
- [Calibrated Language Models Must Hallucinate](https://mlsanity.com/paper/2311.14648/) (2023)
- [End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models](https://mlsanity.com/paper/2205.12487/) (2022)
- [A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4](https://mlsanity.com/paper/2310.12321/) (2023)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [Generating Literal and Implied Subquestions to Fact-check Complex Claims](https://mlsanity.com/paper/2205.06938/) (2022)
- [Generating Fact Checking Briefs](https://mlsanity.com/paper/2011.05448/) (2020)
- [A Retrospective Analysis of the Fake News Challenge Stance Detection Task](https://mlsanity.com/paper/1806.05180/) (2018)
- [The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels](https://mlsanity.com/paper/2405.05860/) (2024)
- [Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL](https://mlsanity.com/paper/2309.06553/) (2023)
- [Controlled Crowdsourcing for High-Quality QA-SRL Annotation](https://mlsanity.com/paper/1911.03243/) (2019)
- [FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation](https://mlsanity.com/paper/2012.15482/) (2020)
- [Explainable Fact-checking through Question Answering](https://mlsanity.com/paper/2110.05369/) (2021)
- [Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation](https://mlsanity.com/paper/2406.03151/) (2024)
- [Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models](https://mlsanity.com/paper/2005.14709/) (2020)
- [Ask to Know More: Generating Counterfactual Explanations for Fake Claims](https://mlsanity.com/paper/2206.04869/) (2022)

## Built on by (most cited first, 2 of 2)

- [FinGrAct: A Framework for FINe-GRrained Evaluation of ACTionability in Explainable Automatic Fact-Checking](https://mlsanity.com/paper/2504.05229/) (2025)
- [Explaining Sources of Uncertainty in Automated Fact-Checking](https://mlsanity.com/paper/2505.17855/) (2025)

## Abstract (arXiv)

In an era increasingly dominated by digital platforms, the spread of misinformation poses a significant challenge, highlighting the need for solutions capable of assessing information veracity. Our research contributes to the field of Explainable Artificial Antelligence (XAI) by developing transformer-based fact-checking models that contextualise and justify their decisions by generating human-accessible explanations. Importantly, we also develop models for automatic evaluation of explanations for fact-checking verdicts across different dimensions such as texttt(self)-contradiction, texttthallucination, textttconvincingness and textttoverall quality. By introducing human-centred evaluation methods and developing specialised datasets, we emphasise the need for aligning Artificial Intelligence (AI)-generated explanations with human judgements. This approach not only advances theoretical knowledge in XAI but also holds practical implications by enhancing the transparency, reliability and users' trust in AI-driven fact-checking systems. Furthermore, the development of our metric learning models is a first step towards potentially increasing efficiency and reducing reliance on extensive manual assessment. Based on experimental results, our best performing generative model textscROUGE-1 score of 47.77, demonstrating superior performance in generating fact-checking explanations, particularly when provided with high-quality evidence. Additionally, the best performing metric learning model showed a moderately strong correlation with human judgements on objective dimensions such as texttt(self)-contradiction and texttthallucination, achieving a Matthews Correlation Coefficient (MCC) of around 0.7.