# Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders

arXiv: https://arxiv.org/abs/2210.05302 · HTML: https://mlsanity.com/paper/2210.05302/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [Whitening Sentence Representations for Better Semantics and Faster Retrieval](https://mlsanity.com/paper/2103.15316/) (2021)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks](https://mlsanity.com/paper/2010.08240/) (2020)
- [Bilateral Multi-Perspective Matching for Natural Language Sentences](https://mlsanity.com/paper/1702.03814/) (2017)
- [On the Sentence Embeddings from Pre-trained Language Models](https://mlsanity.com/paper/2011.05864/) (2020)
- [A Continuously Growing Dataset of Sentential Paraphrases](https://mlsanity.com/paper/1708.00391/) (2017)
- [Text Matching as Image Recognition](https://mlsanity.com/paper/1602.06359/) (2016)
- [Sentence Similarity Learning by Lexical Decomposition and Composition](https://mlsanity.com/paper/1602.07019/) (2016)
- [Transfer Fine-Tuning: A BERT Case Study](https://mlsanity.com/paper/1909.00931/) (2019)
- [Character-based Neural Networks for Sentence Pair Modeling](https://mlsanity.com/paper/1805.08297/) (2018)
- [Explicit Pairwise Word Interaction Modeling Improves Pretrained Transformers for English Semantic Similarity Tasks](https://mlsanity.com/paper/1911.02847/) (2019)

## Built on by (most cited first, 1 of 1)

- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)

## Abstract (arXiv)

Previous works have demonstrated the effectiveness of utilising pre-trained sentence encoders based on their sentence representations for meaning comparison tasks. Though such representations are shown to capture hidden syntax structures, the direct similarity comparison between them exhibits weak sensitivity to word order and structural differences in given sentences. A single similarity score further makes the comparison process hard to interpret. Therefore, we here propose to combine sentence encoders with an alignment component by representing each sentence as a list of predicate-argument spans (where their span representations are derived from sentence encoders), and decomposing the sentence-level meaning comparison into the alignment between their spans for paraphrase identification tasks. Empirical results show that the alignment component brings in both improved performance and interpretability for various sentence encoders. After closer investigation, the proposed approach indicates increased sensitivity to structural difference and enhanced ability to distinguish non-paraphrases with high lexical overlap.