# FarsTail: A Persian Natural Language Inference Dataset

arXiv: https://arxiv.org/abs/2009.08820 · HTML: https://mlsanity.com/paper/2009.08820/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2020.

## Built on by (most cited first, 23 of 23)

- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages](https://mlsanity.com/paper/2104.08726/) (2021)
- [Zero-Shot Tokenizer Transfer](https://mlsanity.com/paper/2405.07883/) (2024)
- [Compositional Evaluation on Japanese Textual Entailment and Similarity](https://mlsanity.com/paper/2208.04826/) (2022)
- [FaBERT: Pre-training BERT on Persian Blogs](https://mlsanity.com/paper/2402.06617/) (2024)
- [FaMTEB: Massive Text Embedding Benchmark in Persian Language](https://mlsanity.com/paper/2502.11571/) (2025)
- [PerCul: A Story-Driven Cultural Evaluation of LLMs in Persian](https://mlsanity.com/paper/2502.07459/) (2025)
- [TookaBERT: A Step Forward for Persian NLU](https://mlsanity.com/paper/2407.16382/) (2024)
- [SILT: Efficient transformer training for inter-lingual inference](https://mlsanity.com/paper/2103.09635/) (2021)
- [Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference](https://mlsanity.com/paper/2106.03983/) (2021)
- [ArNLI: Arabic Natural Language Inference for Entailment and Contradiction Detection](https://mlsanity.com/paper/2209.13953/) (2022)
- [BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching](https://mlsanity.com/paper/2111.02188/) (2021)
- [InferES : A Natural Language Inference Corpus for Spanish Featuring Negation-Based Contrastive and Adversarial Examples](https://mlsanity.com/paper/2210.03068/) (2022)
- [Empowering Persian LLMs for Instruction Following: A Novel Dataset and Training Approach](https://mlsanity.com/paper/2407.11186/) (2024)
- [Hakim: Farsi Text Embedding Model](https://mlsanity.com/paper/2505.08435/) (2025)
- [Contradiction Detection in Persian Text](https://mlsanity.com/paper/2107.01987/) (2021)
- [Unmasking the Factual-Conceptual Gap in Persian Language Models](https://mlsanity.com/paper/2602.17623/) (2026)
- [ViLegalNLI: Natural Language Inference for Vietnamese Legal Texts](https://mlsanity.com/paper/2605.00116/) (2026)
- [Persian Natural Language Inference: A Meta-learning approach](https://mlsanity.com/paper/2205.08755/) (2022)
- [GAPX: Generalized Autoregressive Paraphrase-Identification X](https://mlsanity.com/paper/2210.01979/) (2022)
- [Structural Stress and Learned Helplessness in Afghanistan: A Multi-Layer Analysis of the AFSTRESS Dari Corpus](https://mlsanity.com/paper/2603.27233/) (2026)
- [IHUBERT: Vector-Based Semantic Deduplication and Domain-Balanced Pretraining for Persian Resources](https://mlsanity.com/paper/2606.20089/) (2026)
- [The Annotation Bottleneck in Persian Text NLP: Persian as an Annotation-Scarce Language](https://mlsanity.com/paper/2608.24698/) (2026)

## Abstract (arXiv)

Natural language inference (NLI) is known as one of the central tasks in natural language processing (NLP) which encapsulates many fundamental aspects of language understanding. With the considerable achievements of data-hungry deep learning methods in NLP tasks, a great amount of effort has been devoted to develop more diverse datasets for different languages. In this paper, we present a new dataset for the NLI task in the Persian language, also known as Farsi, which is one of the dominant languages in the Middle East. This dataset, named FarsTail, includes 10,367 samples which are provided in both the Persian language as well as the indexed format to be useful for non-Persian researchers. The samples are generated from 3,539 multiple-choice questions with the least amount of annotator interventions in a way similar to the SciTail dataset. A carefully designed multi-step process is adopted to ensure the quality of the dataset. We also present the results of traditional and state-of-the-art methods on FarsTail including different embedding methods such as word2vec, fastText, ELMo, BERT, and LASER, as well as different modeling approaches such as DecompAtt, ESIM, HBMP, and ULMFiT to provide a solid baseline for the future research. The best obtained test accuracy is 83.38% which shows that there is a big room for improving the current methods to be useful for real-world NLP applications in different languages. We also investigate the extent to which the models exploit superficial clues, also known as dataset biases, in FarsTail, and partition the test set into easy and hard subsets according to the success of biased models. The dataset is available at https://github.com/dml-qom/FarsTail