SemEval-2021 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS)
Not on the map. Why?
Abstract (arXiv)
Understanding tables is an important and relevant task that involves understanding table structure as well as being able to compare and contrast information within cells. In this paper, we address this challenge by presenting a new dataset and tasks that addresses this goal in a shared task in SemEval 2020 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS). Our dataset contains 981 manually-generated tables and an auto-generated dataset of 1980 tables providing over 180K statement and over 16M evidence annotations. SEM-TAB-FACTS featured two sub-tasks. In sub-task A, the goal was to determine if a statement is supported, refuted or unknown in relation to a table. In sub-task B, the focus was on identifying the specific cells of a table that provide evidence for the statement. 69 teams signed up to participate in the task with 19 successful submissions to subtask A and 12 successful submissions to subtask B. We present our results and main findings from the competition.
Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 12 of its 25 references that are in the map. First posted on arXiv 28 May 2021.
Builds on
12 of 25 references.
References · 12
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2015A large annotated corpus for learning natural language inferencecited by 1,837
- 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
- 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
- 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
- 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
- 2020INFOTABS: Inference on Tables as Semi-structured Datacited by 84
- 2019SemEval-2019 Task 8: Fact Checking in Community Question Answering Forumscited by 18
- 2021TAPAS at SemEval-2021 Task 9: Reasoning over tables with intermediate pre-trainingcited by 14
- 2021BreakingBERT@IITK at SemEval-2021 Task 9 : Statement Verification and Evidence Finding with Tablescited by 1
Built on by
Citing papers per year: 2021: 4; 2022: 4; 2023: 6; 2024: 8; 2025: 8; 2026: 5.
35 papers.
Sort:
By area:
- 2021A Survey on Automated Fact-Checkingcited by 376
- 2023FactKG: Fact Verification via Reasoning on Knowledge Graphscited by 57
- 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tablescited by 51
- 2022PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-trainingcited by 41
- 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checkingcited by 34
- 2023TempTabQA: Temporal Question Answering for Semi-Structured Tablescited by 27
- 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23
- 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
- 2024GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answeringcited by 18
- 2022Table-based Fact Verification with Self-adaptive Mixture of Expertscited by 12
- 2024ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphscited by 11
- 2025CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?cited by 11
23 more
- 2025SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrievalcited by 11
- 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoningcited by 10
- 2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challengescited by 10
- 2021Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoningcited by 9
- 2024Knowledge-Aware Reasoning over Multimodal Semi-structured Tablescited by 9
- 2025MuSciClaims: Multimodal Scientific Claim Verificationcited by 8
- 2022Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Frameworkcited by 6
- 2023Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Setscited by 6
- 2026SciClaimEval: Cross-modal Claim Verification in Scientific Paperscited by 6
- 2021Logic-level Evidence Retrieval and Graph-based Verification Network for Table-based Fact Verificationcited by 4
- 2025Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Paperscited by 4
- 2022Leveraging Data Recasting to Enhance Tabular Reasoningcited by 3
- 2023INO at Factify 2: Structure Coherence based Multi-Modal Fact Verificationcited by 3
- 2024EHRCon: Dataset for Checking Consistency between Unstructured Notes and Structured Tables in Electronic Health Recordscited by 3
- 2025Task-Oriented Automatic Fact-Checking with Frame-Semanticscited by 3
- 2026ClaimDB: A Fact Verification Benchmark over Large Structured Datacited by 3
- 2021TabPert: An Effective Platform for Tabular Perturbationcited by 2
- 2025What to Keep and What to Drop: Adaptive Table Filtering Frameworkcited by 1
- 2025PanelTR: Zero-Shot Table Reasoning Framework Through Multi-Agent Scientific Discussioncited by 1
- 2026Analyzing LLM Instruction Optimization for Tabular Fact Verificationcited by 1
- 2024Bridging Textual and Tabular Worlds for Fact Verification: A Lightweight, Attention-Based Modelcited by 0
- 2026Frame-Guided Synthetic Claim Generation for Automatic Fact-Checking Using High-Volume Tabular Datacited by 0
- 2026A Multi-Agent Approach for Claim Verification from Tabular Data Documentscited by 0
- 2026A Multi-Agent Approach for Claim Verification from Tabular Data Documents19 Apr 2026
- 2026Analyzing LLM Instruction Optimization for Tabular Fact Verification20 Feb 2026
- 2026SciClaimEval: Cross-modal Claim Verification in Scientific Papers7 Feb 2026
- 2026Frame-Guided Synthetic Claim Generation for Automatic Fact-Checking Using High-Volume Tabular Data23 Jan 2026
- 2026ClaimDB: A Fact Verification Benchmark over Large Structured Data21 Jan 2026
- 2025PanelTR: Zero-Shot Table Reasoning Framework Through Multi-Agent Scientific Discussion8 Aug 2025
- 2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challenges31 Jul 2025
- 2025What to Keep and What to Drop: Adaptive Table Filtering Framework30 Jun 2025
- 2025Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers12 Jun 2025
- 2025MuSciClaims: Multimodal Scientific Claim Verification5 Jun 2025
- 2025SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval15 May 2025
- 2025CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?27 Mar 2025
23 more
- 2025Task-Oriented Automatic Fact-Checking with Frame-Semantics23 Jan 2025
- 2024GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answering2 Dec 2024
- 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning18 Sep 2024
- 2024Claim Verification in the Age of Large Language Models: A Survey26 Aug 2024
- 2024Knowledge-Aware Reasoning over Multimodal Semi-structured Tables25 Aug 2024
- 2024EHRCon: Dataset for Checking Consistency between Unstructured Notes and Structured Tables in Electronic Health Records24 Jun 2024
- 2024Bridging Textual and Tabular Worlds for Fact Verification: A Lightweight, Attention-Based Model26 Mar 2024
- 2024ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphs12 Mar 2024
- 2024CABINET: Content Relevance based Noise Reduction for Table Question Answering2 Feb 2024
- 2023Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets15 Nov 2023
- 2023TempTabQA: Temporal Question Answering for Semi-Structured Tables14 Nov 2023
- 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables22 May 2023
- 2023FactKG: Fact Verification via Reasoning on Knowledge Graphs11 May 2023
- 2023INO at Factify 2: Structure Coherence based Multi-Modal Fact Verification2 Mar 2023
- 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking27 Jan 2023
- 2022Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Framework20 Dec 2022
- 2022Leveraging Data Recasting to Enhance Tabular Reasoning23 Nov 2022
- 2022PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training5 Nov 2022
- 2022Table-based Fact Verification with Self-adaptive Mixture of Experts19 Apr 2022
- 2021Logic-level Evidence Retrieval and Graph-based Verification Network for Table-based Fact Verification14 Sep 2021
- 2021A Survey on Automated Fact-Checking26 Aug 2021
- 2021TabPert: An Effective Platform for Tabular Perturbation2 Aug 2021
- 2021Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning2 Aug 2021
- 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tablescited by 51
- 2024GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answeringcited by 18
- 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoningcited by 10
- 2025What to Keep and What to Drop: Adaptive Table Filtering Frameworkcited by 1
- 2025PanelTR: Zero-Shot Table Reasoning Framework Through Multi-Agent Scientific Discussioncited by 1
- 2026Analyzing LLM Instruction Optimization for Tabular Fact Verificationcited by 1
- 2026Analyzing LLM Instruction Optimization for Tabular Fact Verification20 Feb 2026
- 2025PanelTR: Zero-Shot Table Reasoning Framework Through Multi-Agent Scientific Discussion8 Aug 2025
- 2025What to Keep and What to Drop: Adaptive Table Filtering Framework30 Jun 2025
- 2024GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answering2 Dec 2024
- 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning18 Sep 2024
- 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables22 May 2023
- 2021A Survey on Automated Fact-Checkingcited by 376
- 2023FactKG: Fact Verification via Reasoning on Knowledge Graphscited by 57
- 2022PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-trainingcited by 41
- 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checkingcited by 34
- 2023TempTabQA: Temporal Question Answering for Semi-Structured Tablescited by 27
- 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23
- 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
- 2024ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphscited by 11
- 2025CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?cited by 11
- 2025SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrievalcited by 11
- 2021Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoningcited by 9
- 2024Knowledge-Aware Reasoning over Multimodal Semi-structured Tablescited by 9
12 most cited of 26 All 26 (JSON)
- 2026SciClaimEval: Cross-modal Claim Verification in Scientific Papers7 Feb 2026
- 2026Frame-Guided Synthetic Claim Generation for Automatic Fact-Checking Using High-Volume Tabular Data23 Jan 2026
- 2026ClaimDB: A Fact Verification Benchmark over Large Structured Data21 Jan 2026
- 2025Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers12 Jun 2025
- 2025MuSciClaims: Multimodal Scientific Claim Verification5 Jun 2025
- 2025SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval15 May 2025
- 2025CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?27 Mar 2025
- 2025Task-Oriented Automatic Fact-Checking with Frame-Semantics23 Jan 2025
- 2024Claim Verification in the Age of Large Language Models: A Survey26 Aug 2024
- 2024Knowledge-Aware Reasoning over Multimodal Semi-structured Tables25 Aug 2024
- 2024EHRCon: Dataset for Checking Consistency between Unstructured Notes and Structured Tables in Electronic Health Records24 Jun 2024
- 2024Bridging Textual and Tabular Worlds for Fact Verification: A Lightweight, Attention-Based Model26 Mar 2024
12 newest of 26 All 26 (JSON)