# I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviews

arXiv: https://arxiv.org/abs/2104.06893 · HTML: https://mlsanity.com/paper/2104.06893/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 56 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Breaking the Softmax Bottleneck: A High-Rank RNN Language Model](https://mlsanity.com/paper/1711.03953/) (2017)
- [Debiasing Pre-trained Contextualised Embeddings](https://mlsanity.com/paper/2101.09523/) (2021)
- [Challenges of Using Text Classifiers for Causal Inference](https://mlsanity.com/paper/1810.00956/) (2018)
- [SemEval-2020 Task 5: Counterfactual Recognition](https://mlsanity.com/paper/2008.00563/) (2020)
- [Think Globally, Embed Locally --- Locally Linear Meta-embedding of Words](https://mlsanity.com/paper/1709.06671/) (2017)
- [BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models](https://mlsanity.com/paper/2007.14128/) (2020)
- [Language-Independent Tokenisation Rivals Language-Specific Tokenisation for Word Similarity Prediction](https://mlsanity.com/paper/2002.11004/) (2020)
- [IITK-RSA at SemEval-2020 Task 5: Detecting Counterfactuals](https://mlsanity.com/paper/2007.10866/) (2020)
- [ISCAS at SemEval-2020 Task 5: Pre-trained Transformers for Counterfactual Statement Modeling](https://mlsanity.com/paper/2009.08171/) (2020)

## Built on by (most cited first, 30 of 45)

- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Efficient Few-Shot Learning Without Prompts](https://mlsanity.com/paper/2209.11055/) (2022)
- [Making Text Embedders Few-Shot Learners](https://mlsanity.com/paper/2409.15700/) (2024)
- [RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledge](https://mlsanity.com/paper/2311.08147/) (2023)
- [Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks](https://mlsanity.com/paper/2511.07025/) (2025)
- [KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model](https://mlsanity.com/paper/2506.20923/) (2025)
- [Position: Measure Dataset Diversity, Don't Just Claim It](https://mlsanity.com/paper/2407.08188/) (2024)
- [Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack](https://mlsanity.com/paper/2407.16695/) (2024)
- [F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World](https://mlsanity.com/paper/2603.19223/) (2026)
- [Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings](https://mlsanity.com/paper/2506.01435/) (2025)
- [Automated Few-shot Classification with Instruction-Finetuned Language Models](https://mlsanity.com/paper/2305.12576/) (2023)
- [Randomly Removing 50% of Dimensions in Text Embeddings has Minimal Impact on Retrieval and Classification Tasks](https://mlsanity.com/paper/2508.17744/) (2025)
- [F2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source Data](https://mlsanity.com/paper/2510.02294/) (2025)
- [Don't Reinvent the Wheel: Efficient Instruction-Following Text Embedding based on Guided Space Transformation](https://mlsanity.com/paper/2505.24754/) (2025)
- [Enhancing Lexicon-Based Text Embeddings with Large Language Models](https://mlsanity.com/paper/2501.09749/) (2025)
- [CSRv2: Unlocking Ultra-Sparse Embeddings](https://mlsanity.com/paper/2602.05735/) (2026)
- [Bilingual BSARD: Extending Statutory Article Retrieval to Dutch](https://mlsanity.com/paper/2412.07462/) (2024)
- [New Encoders for German Trained from Scratch: Comparing ModernGBERT with Converted LLM2Vec Models](https://mlsanity.com/paper/2505.13136/) (2025)
- [The Embedder's Dilemma: LLMs Are Better, but at What Cost?](https://mlsanity.com/paper/2608.12875/) (2026)
- [DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning](https://mlsanity.com/paper/2401.13621/) (2024)
- [Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings](https://mlsanity.com/paper/2509.12892/) (2025)
- [AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification](https://mlsanity.com/paper/2311.00408/) (2023)
- [LGAI-EMBEDDING-Preview Technical Report](https://mlsanity.com/paper/2506.07438/) (2025)
- [On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey](https://mlsanity.com/paper/2507.20783/) (2025)
- [PTEB: Towards Robust Text Embedding Evaluation via Stochastic Paraphrasing at Evaluation Time with LLMs](https://mlsanity.com/paper/2510.06730/) (2025)
- [One prompt is not enough: Instruction Sensitivity Undermines Embedding Model Evaluation](https://mlsanity.com/paper/2605.22544/) (2026)
- [Uncovering the Latent Potential of Deep Intermediate Representations](https://mlsanity.com/paper/2605.23033/) (2026)
- [Non-contrastive sentence representations via self-supervision](https://mlsanity.com/paper/2310.17690/) (2023)
- [Assessing Logical Reasoning Capabilities of Encoder-Only Transformer Models](https://mlsanity.com/paper/2312.11720/) (2023)

## Abstract (arXiv)

Counterfactual statements describe events that did not or cannot take place. We consider the problem of counterfactual detection (CFD) in product reviews. For this purpose, we annotate a multilingual CFD dataset from Amazon product reviews covering counterfactual statements written in English, German, and Japanese languages. The dataset is unique as it contains counterfactuals in multiple languages, covers a new application area of e-commerce reviews, and provides high quality professional annotations. We train CFD models using different text representation methods and classifiers. We find that these models are robust against the selectional biases introduced due to cue phrase-based sentence selection. Moreover, our CFD dataset is compatible with prior datasets and can be merged to learn accurate CFD models. Applying machine translation on English counterfactual examples to create multilingual data performs poorly, demonstrating the language-specificity of this problem, which has been ignored so far.