# DGA-Net Dynamic Gaussian Attention Network for Sentence Semantic Matching

arXiv: https://arxiv.org/abs/2106.04905 · HTML: https://mlsanity.com/paper/2106.04905/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [A Decomposable Attention Model for Natural Language Inference](https://mlsanity.com/paper/1606.01933/) (2016)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Bilateral Multi-Perspective Matching for Natural Language Sentences](https://mlsanity.com/paper/1702.03814/) (2017)
- [DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding](https://mlsanity.com/paper/1709.04696/) (2017)
- [Learning to Compose Task-Specific Tree Structures](https://mlsanity.com/paper/1707.02786/) (2017)
- [Natural Language Inference by Tree-Based Convolution and Heuristic Matching](https://mlsanity.com/paper/1512.08422/) (2015)
- [Joint Item Recommendation and Attribute Inference: An Adaptive Graph Convolutional Network Approach](https://mlsanity.com/paper/2005.12021/) (2020)
- [Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information](https://mlsanity.com/paper/1805.11360/) (2018)
- [Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention](https://mlsanity.com/paper/1605.09090/) (2016)
- [Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering](https://mlsanity.com/paper/1806.04330/) (2018)
- [Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding](https://mlsanity.com/paper/1808.07383/) (2018)
- [Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference](https://mlsanity.com/paper/1708.01353/) (2017)
- [Distance-based Self-Attention Network for Natural Language Inference](https://mlsanity.com/paper/1712.02047/) (2017)
- [R2-Net: Relation of Relation Learning Network for Sentence Semantic Matching](https://mlsanity.com/paper/2012.08920/) (2020)

## Abstract (arXiv)

Sentence semantic matching requires an agent to determine the semantic relation between two sentences, where much recent progress has been made by the advancement of representation learning techniques and inspiration of human behaviors. Among all these methods, attention mechanism plays an essential role by selecting important parts effectively. However, current attention methods either focus on all the important parts in a static way or only select one important part at one attention step dynamically, which leaves a large space for further improvement. To this end, in this paper, we design a novel Dynamic Gaussian Attention Network (DGA-Net) to combine the advantages of current static and dynamic attention methods. More specifically, we first leverage pre-trained language model to encode the input sentences and construct semantic representations from a global perspective. Then, we develop a Dynamic Gaussian Attention (DGA) to dynamically capture the important parts and corresponding local contexts from a detailed perspective. Finally, we combine the global information and detailed local information together to decide the semantic relation of sentences comprehensively and precisely. Extensive experiments on two popular sentence semantic matching tasks demonstrate that our proposed DGA-Net is effective in improving the ability of attention mechanism.