# A Multi-Level Attention Model for Evidence-Based Fact Checking

arXiv: https://arxiv.org/abs/2106.00950 · HTML: https://mlsanity.com/paper/2106.00950/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [Combining Fact Extraction and Verification with Neural Semantic Matching Networks](https://mlsanity.com/paper/1811.07039/) (2018)
- [Fine-grained Fact Verification with Kernel Graph Attention Network](https://mlsanity.com/paper/1910.09796/) (2019)
- [GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification](https://mlsanity.com/paper/1908.01843/) (2019)
- [Coreferential Reasoning Learning for Language Representation](https://mlsanity.com/paper/2004.06870/) (2020)
- [Reasoning Over Semantic-Level Graph for Fact Checking](https://mlsanity.com/paper/1909.03745/) (2019)
- [UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification](https://mlsanity.com/paper/1809.01479/) (2018)
- [BERT for Evidence Retrieval and Claim Verification](https://mlsanity.com/paper/1910.02655/) (2019)
- [Revealing the Importance of Semantic Retrieval for Machine Reading at Scale](https://mlsanity.com/paper/1909.08041/) (2019)
- [Modeling Localness for Self-Attention Networks](https://mlsanity.com/paper/1810.10182/) (2018)
- [A Paragraph-level Multi-task Learning Model for Scientific Fact-Verification](https://mlsanity.com/paper/2012.14500/) (2020)
- [DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking](https://mlsanity.com/paper/2004.12864/) (2020)
- [TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification](https://mlsanity.com/paper/1808.03465/) (2018)
- [Team Papelo: Transformer Networks at FEVER](https://mlsanity.com/paper/1901.02534/) (2019)
- [Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verification](https://mlsanity.com/paper/2010.05111/) (2020)

## Built on by (most cited first, 13 of 13)

- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Internet-augmented language models through few-shot prompting for open-domain question answering](https://mlsanity.com/paper/2203.05115/) (2022)
- [Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method](https://mlsanity.com/paper/2310.00305/) (2023)
- [Synthetic Disinformation Attacks on Automated Fact Verification Systems](https://mlsanity.com/paper/2202.09381/) (2022)
- [EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification](https://mlsanity.com/paper/2310.09754/) (2023)
- [BiDeV: Bilateral Defusing Verification for Complex Claim Fact-Checking](https://mlsanity.com/paper/2502.16181/) (2025)
- [Adversarial Attacks Against Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2509.08463/) (2025)
- [XFEVER: Exploring Fact Verification across Languages](https://mlsanity.com/paper/2310.16278/) (2023)
- [IMCI: Integrate Multi-view Contextual Information for Fact Extraction and Verification](https://mlsanity.com/paper/2208.14001/) (2022)
- [Robust Deepfake On Unrestricted Media: Generation And Detection](https://mlsanity.com/paper/2202.06228/) (2022)
- [Multi-Evidence based Fact Verification via A Confidential Graph Neural Network](https://mlsanity.com/paper/2405.10481/) (2024)
- [Reasoning Paths as Signals: Augmenting Multi-hop Fact Verification through Structural Reasoning Progression](https://mlsanity.com/paper/2506.07075/) (2025)
- [Bridging Textual and Tabular Worlds for Fact Verification: A Lightweight, Attention-Based Model](https://mlsanity.com/paper/2403.17361/) (2024)

## Abstract (arXiv)

Evidence-based fact checking aims to verify the truthfulness of a claim against evidence extracted from textual sources. Learning a representation that effectively captures relations between a claim and evidence can be challenging. Recent state-of-the-art approaches have developed increasingly sophisticated models based on graph structures. We present a simple model that can be trained on sequence structures. Our model enables inter-sentence attentions at different levels and can benefit from joint training. Results on a large-scale dataset for Fact Extraction and VERification (FEVER) show that our model outperforms the graph-based approaches and yields 1.09% and 1.42% improvements in label accuracy and FEVER score, respectively, over the best published model.