# BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context

arXiv: https://arxiv.org/abs/2009.12452 · HTML: https://mlsanity.com/paper/2009.12452/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Data Augmentation for Mental Health Classification on Social Media](https://mlsanity.com/paper/2112.10064/) (2021)
- [Paraphrasing Techniques for Maritime QA system](https://mlsanity.com/paper/2203.10854/) (2022)
- [Explainable AI: XAI-Guided Context-Aware Data Augmentation](https://mlsanity.com/paper/2506.03484/) (2025)
- [Data Augmentation using Transformers and Similarity Measures for Improving Arabic Text Classification](https://mlsanity.com/paper/2212.13939/) (2022)
- [Mitigating Data Scarcity for Large Language Models](https://mlsanity.com/paper/2302.01806/) (2023)
- [On Evaluation Protocols for Data Augmentation in a Limited Data Scenario](https://mlsanity.com/paper/2402.14895/) (2024)
- [Semantic Parsing in Limited Resource Conditions](https://mlsanity.com/paper/2309.07429/) (2023)
- [A Persuasion-Based Prompt Learning Approach to Improve Smishing Detection through Data Augmentation](https://mlsanity.com/paper/2411.02403/) (2024)

## Abstract (arXiv)

Newly-introduced deep learning architectures, namely BERT, XLNet, RoBERTa and ALBERT, have been proved to be robust on several NLP tasks. However, the datasets trained on these architectures are fixed in terms of size and generalizability. To relieve this issue, we apply one of the most inexpensive solutions to update these datasets. We call this approach BET by which we analyze the backtranslation data augmentation on the transformer-based architectures. Using the Google Translate API with ten intermediary languages from ten different language families, we externally evaluate the results in the context of automatic paraphrase identification in a transformer-based framework. Our findings suggest that BET improves the paraphrase identification performance on the Microsoft Research Paraphrase Corpus (MRPC) to more than 3% on both accuracy and F1 score. We also analyze the augmentation in the low-data regime with downsampled versions of MRPC, Twitter Paraphrase Corpus (TPC) and Quora Question Pairs. In many low-data cases, we observe a switch from a failing model on the test set to reasonable performances. The results demonstrate that BET is a highly promising data augmentation technique: to push the current state-of-the-art of existing datasets and to bootstrap the utilization of deep learning architectures in the low-data regime of a hundred samples.