# Self-Attention Neural Bag-of-Features

arXiv: https://arxiv.org/abs/2201.11092 · HTML: https://mlsanity.com/paper/2201.11092/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 26 Jan 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Bag of Color Features For Color Constancy](https://mlsanity.com/paper/1906.04445/) (2019)
- [Temporal Logistic Neural Bag-of-Features for Financial Time series Forecasting leveraging Limit Order Book Data](https://mlsanity.com/paper/1901.08280/) (2019)
- [Attention-based Neural Bag-of-Features Learning for Sequence Data](https://mlsanity.com/paper/2005.12250/) (2020)
- [Learning to ignore: rethinking attention in CNNs](https://mlsanity.com/paper/2111.05684/) (2021)

## Built on by (most cited first, 1 of 1)

- [Efficient CNN with uncorrelated Bag of Features pooling](https://mlsanity.com/paper/2209.10865/) (2022)

## Abstract (arXiv)

In this work, we propose several attention formulations for multivariate sequence data. We build on top of the recently introduced 2D-Attention and reformulate the attention learning methodology by quantifying the relevance of feature/temporal dimensions through latent spaces based on self-attention rather than learning them directly. In addition, we propose a joint feature-temporal attention mechanism that learns a joint 2D attention mask highlighting relevant information without treating feature and temporal representations independently. The proposed approaches can be used in various architectures and we specifically evaluate their application together with Neural Bag of Features feature extraction module. Experiments on several sequence data analysis tasks show the improved performance yielded by our approach compared to standard methods.