# R-DEIM Net: An Efficient Rationale-Augmented Dual-Expert Interaction Model for Paraphrase Detection

arXiv: https://arxiv.org/abs/2609.30100 · HTML: https://mlsanity.com/paper/2609.30100/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 60 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [MMTEB: Massive Multilingual Text Embedding Benchmark](https://mlsanity.com/paper/2502.13595/) (2025)
- [Bilateral Multi-Perspective Matching for Natural Language Sentences](https://mlsanity.com/paper/1702.03814/) (2017)
- [Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning](https://mlsanity.com/paper/1804.00079/) (2018)
- [Natural Language Understanding with the Quora Question Pairs Dataset](https://mlsanity.com/paper/1907.01041/) (2019)
- [ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs](https://mlsanity.com/paper/1512.05193/) (2015)
- [Natural Language Inference over Interaction Space](https://mlsanity.com/paper/1709.04348/) (2017)
- [Adversarial Domain Adaptation for Duplicate Question Detection](https://mlsanity.com/paper/1809.02255/) (2018)
- [Sentence Similarity Learning by Lexical Decomposition and Composition](https://mlsanity.com/paper/1602.07019/) (2016)
- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [Neural Paraphrase Identification of Questions with Noisy Pretraining](https://mlsanity.com/paper/1704.04565/) (2017)
- [Shortcut-Stacked Sentence Encoders for Multi-Domain Inference](https://mlsanity.com/paper/1708.02312/) (2017)
- [REGMAPR - Text Matching Made Easy](https://mlsanity.com/paper/1808.04343/) (2018)
- [Cell-aware Stacked LSTMs for Modeling Sentences](https://mlsanity.com/paper/1809.02279/) (2018)
- [Identifying Semantically Duplicate Questions Using Data Science Approach: A Quora Case Study](https://mlsanity.com/paper/2004.11694/) (2020)

## Abstract (arXiv)

Recent advances in paraphrase detection reveal a fundamental trade-off: large language models achieve high accuracy but require high computation, while efficient Siamese-BERT variants offer practical scalability with reduced transparency in rationale generation. We present R-DEIM Net, a 76M-parameter dual-expert architecture exploring whether moderate-scale models can achieve competitive accuracy on paraphrase detection while enabling human-readable rationale generation. The architecture combines two specialized components: an Interaction Expert that captures token-level similarity patterns through multi-scale 2D convolutions and attention head allowing variable input length, and a Reasoning Expert that uses a Flan-T5-small decoder to generate rationales as auxiliary supervision. Rather than re-encoding generated text, we extract and pool decoder hidden states as complementary features for classification. On the Quora Question Pairs dataset, R-DEIM Net achieves 90.07% accuracy and 90.16% F1-score via 10-fold cross-validation. This represents competitive performance with strong transformer-based baselines (e.g., MFAE BERT: 90.54% accuracy) and recent large language model based approaches (LLaMA-70B) while using a substantially smaller parameter budget. The model generates rationales alongside predictions, providing potential for auxiliary human-readable descriptions.