# Table-based Fact Verification with Self-adaptive Mixture of Experts

arXiv: https://arxiv.org/abs/2204.08753 · HTML: https://mlsanity.com/paper/2204.08753/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Learning Factored Representations in a Deep Mixture of Experts](https://mlsanity.com/paper/1312.4314/) (2013)
- [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://mlsanity.com/paper/2107.07653/) (2021)
- [Assessing The Factual Accuracy of Generated Text](https://mlsanity.com/paper/1905.13322/) (2019)
- [INFOTABS: Inference on Tables as Semi-structured Data](https://mlsanity.com/paper/2005.06117/) (2020)
- [Understanding tables with intermediate pre-training](https://mlsanity.com/paper/2010.00571/) (2020)
- [SemEval-2021 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS)](https://mlsanity.com/paper/2105.13995/) (2021)
- [Program Enhanced Fact Verification with Verbalization and Graph Attention Network](https://mlsanity.com/paper/2010.03084/) (2020)
- [LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network](https://mlsanity.com/paper/2004.13659/) (2020)
- [Do Sentence Interactions Matter? Leveraging Sentence Level Representations for Fake News Classification](https://mlsanity.com/paper/1910.12203/) (2019)
- [Exploring Decomposition for Table-based Fact Verification](https://mlsanity.com/paper/2109.11020/) (2021)
- [LSTM-based Mixture-of-Experts for Knowledge-Aware Dialogues](https://mlsanity.com/paper/1605.01652/) (2016)

## Built on by (most cited first, 12 of 12)

- [Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning](https://mlsanity.com/paper/2301.13808/) (2023)
- [SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables](https://mlsanity.com/paper/2305.13186/) (2023)
- [PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training](https://mlsanity.com/paper/2211.02816/) (2022)
- [MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering](https://mlsanity.com/paper/2305.12820/) (2023)
- [From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning](https://mlsanity.com/paper/2304.07995/) (2023)
- [UniRPG: Unified Discrete Reasoning over Table and Text as Program Generation](https://mlsanity.com/paper/2210.08249/) (2022)
- [TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning](https://mlsanity.com/paper/2409.11724/) (2024)
- [MFORT-QA: Multi-hop Few-shot Open Rich Table Question Answering](https://mlsanity.com/paper/2403.19116/) (2024)
- [TACR: A Table-alignment-based Cell-selection and Reasoning Model for Hybrid Question-Answering](https://mlsanity.com/paper/2305.14682/) (2023)
- [Are Large Language Models Table-based Fact-Checkers?](https://mlsanity.com/paper/2402.02549/) (2024)
- [SCFCRC: Simultaneously Counteract Feature Camouflage and Relation Camouflage for Fraud Detection](https://mlsanity.com/paper/2501.12430/) (2025)
- [FGTR: Fine-Grained Multi-Table Retrieval via Hierarchical LLM Reasoning](https://mlsanity.com/paper/2603.12702/) (2026)

## Abstract (arXiv)

The table-based fact verification task has recently gained widespread attention and yet remains to be a very challenging problem. It inherently requires informative reasoning over natural language together with different numerical and logical reasoning on tables (e.g., count, superlative, comparative). Considering that, we exploit mixture-of-experts and present in this paper a new method: Self-adaptive Mixture-of-Experts Network (SaMoE). Specifically, we have developed a mixture-of-experts neural network to recognize and execute different types of reasoning -- the network is composed of multiple experts, each handling a specific part of the semantics for reasoning, whereas a management module is applied to decide the contribution of each expert network to the verification result. A self-adaptive method is developed to teach the management module combining results of different experts more efficiently without external knowledge. The experimental results illustrate that our framework achieves 85.1% accuracy on the benchmark dataset TabFact, comparable with the previous state-of-the-art models. We hope our framework can serve as a new baseline for table-based verification. Our code is available at https://github.com/THUMLP/SaMoE.