# Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning

arXiv: https://arxiv.org/abs/2106.00248 · HTML: https://mlsanity.com/paper/2106.00248/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [A Corpus for Reasoning About Natural Language Grounded in Photographs](https://mlsanity.com/paper/1811.00491/) (2018)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [INFOTABS: Inference on Tables as Semi-structured Data](https://mlsanity.com/paper/2005.06117/) (2020)
- [Understanding tables with intermediate pre-training](https://mlsanity.com/paper/2010.00571/) (2020)
- [Program Enhanced Fact Verification with Verbalization and Graph Attention Network](https://mlsanity.com/paper/2010.03084/) (2020)
- [LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network](https://mlsanity.com/paper/2004.13659/) (2020)
- [Incorporating External Knowledge to Enhance Tabular Reasoning](https://mlsanity.com/paper/2104.04243/) (2021)

## Built on by (most cited first, 3 of 3)

- [PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training](https://mlsanity.com/paper/2211.02816/) (2022)
- [Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Framework](https://mlsanity.com/paper/2212.10097/) (2022)
- [A Multi-Agent Approach for Claim Verification from Tabular Data Documents](https://mlsanity.com/paper/2604.17225/) (2026)

## Abstract (arXiv)

Tables are widely used in various kinds of documents to present information concisely. Understanding tables is a challenging problem that requires an understanding of language and table structure, along with numerical and logical reasoning. In this paper, we present our systems to solve Task 9 of SemEval-2021: Statement Verification and Evidence Finding with Tables (SEM-TAB-FACTS). The task consists of two subtasks: (A) Given a table and a statement, predicting whether the table supports the statement and (B) Predicting which cells in the table provide evidence for/against the statement. We fine-tune TAPAS (a model which extends BERT's architecture to capture tabular structure) for both the subtasks as it has shown state-of-the-art performance in various table understanding tasks. In subtask A, we evaluate how transfer learning and standardizing tables to have a single header row improves TAPAS' performance. In subtask B, we evaluate how different fine-tuning strategies can improve TAPAS' performance. Our systems achieve an F1 score of 67.34 in subtask A three-way classification, 72.89 in subtask A two-way classification, and 62.95 in subtask B.