# Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding

arXiv: https://arxiv.org/abs/1808.07383 · HTML: https://mlsanity.com/paper/1808.07383/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2018.

## Built on by (most cited first, 8 of 8)

- [Boosting Crowd Counting via Multifaceted Attention](https://mlsanity.com/paper/2203.02636/) (2022)
- [A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions](https://mlsanity.com/paper/2208.13629/) (2022)
- [MAG-SQL: Multi-Agent Generative Approach with Soft Schema Linking and Iterative Sub-SQL Refinement for Text-to-SQL](https://mlsanity.com/paper/2408.07930/) (2024)
- [Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy Logic](https://mlsanity.com/paper/2109.08927/) (2021)
- [On the Security Vulnerabilities of Text-to-SQL Models](https://mlsanity.com/paper/2211.15363/) (2022)
- [LadRa-Net: Locally-Aware Dynamic Re-read Attention Net for Sentence Semantic Matching](https://mlsanity.com/paper/2108.02915/) (2021)
- [DGA-Net Dynamic Gaussian Attention Network for Sentence Semantic Matching](https://mlsanity.com/paper/2106.04905/) (2021)
- [Meta-aware Learning in text-to-SQL Large Language Model](https://mlsanity.com/paper/2505.18929/) (2025)

## Abstract (arXiv)

In this paper, we propose Dynamic Self-Attention (DSA), a new self-attention mechanism for sentence embedding. We design DSA by modifying dynamic routing in capsule network (Sabouretal.,2017) for natural language processing. DSA attends to informative words with a dynamic weight vector. We achieve new state-of-the-art results among sentence encoding methods in Stanford Natural Language Inference (SNLI) dataset with the least number of parameters, while showing comparative results in Stanford Sentiment Treebank (SST) dataset.