# Baselines and test data for cross-lingual inference

arXiv: https://arxiv.org/abs/1704.05347 · HTML: https://mlsanity.com/paper/1704.05347/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2017.

## Built on by (most cited first, 11 of 11)

- [IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages](https://mlsanity.com/paper/2201.11732/) (2022)
- [Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training](https://mlsanity.com/paper/2206.00621/) (2022)
- [ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation](https://mlsanity.com/paper/2211.04861/) (2022)
- [MVL-SIB: A Massively Multilingual Vision-Language Benchmark for Cross-Modal Topical Matching](https://mlsanity.com/paper/2502.12852/) (2025)
- [XNLI 2.0: Improving XNLI dataset and performance on Cross Lingual Understanding (XLU)](https://mlsanity.com/paper/2301.06527/) (2023)
- [RC3: Regularized Contrastive Cross-lingual Cross-modal Pre-training](https://mlsanity.com/paper/2305.07927/) (2023)
- [Survey of NLU Benchmarks Diagnosing Linguistic Phenomena: Why not Standardize Diagnostics Benchmarks?](https://mlsanity.com/paper/2507.20419/) (2025)
- [Multilingual Vision-Language Models, A Survey](https://mlsanity.com/paper/2509.22123/) (2025)
- [Improving Natural Language Inference in Arabic using Transformer Models and Linguistically Informed Pre-Training](https://mlsanity.com/paper/2307.14666/) (2023)
- [E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets](https://mlsanity.com/paper/2609.11334/) (2026)
- [E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding](https://mlsanity.com/paper/2609.33530/) (2026)

## Abstract (arXiv)

The recent years have seen a revival of interest in textual entailment, sparked by i) the emergence of powerful deep neural network learners for natural language processing and ii) the timely development of large-scale evaluation datasets such as SNLI. Recast as natural language inference, the problem now amounts to detecting the relation between pairs of statements: they either contradict or entail one another, or they are mutually neutral. Current research in natural language inference is effectively exclusive to English. In this paper, we propose to advance the research in SNLI-style natural language inference toward multilingual evaluation. To that end, we provide test data for four major languages: Arabic, French, Spanish, and Russian. We experiment with a set of baselines. Our systems are based on cross-lingual word embeddings and machine translation. While our best system scores an average accuracy of just over 75%, we focus largely on enabling further research in multilingual inference.