# Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

arXiv: https://arxiv.org/abs/1512.08756 · HTML: https://mlsanity.com/paper/1512.08756/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2015.

## Built on by (most cited first, 23 of 23)

- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [A Systematic Review of Deep Learning-based Research on Radiology Report Generation](https://mlsanity.com/paper/2311.14199/) (2023)
- [Event-Based Contrastive Learning for Medical Time Series](https://mlsanity.com/paper/2312.10308/) (2023)
- [ArmanEmo: A Persian Dataset for Text-based Emotion Detection](https://mlsanity.com/paper/2207.11808/) (2022)
- [Are the Multilingual Models Better? Improving Czech Sentiment with Transformers](https://mlsanity.com/paper/2108.10640/) (2021)
- [Description-based Controllable Text-to-Speech with Cross-Lingual Voice Control](https://mlsanity.com/paper/2409.17452/) (2024)
- [Weakly Supervised Attention-based Models Using Activation Maps for Citrus Mite and Insect Pest Classification](https://mlsanity.com/paper/2110.00881/) (2021)
- [Voice Impression Control in Zero-Shot TTS](https://mlsanity.com/paper/2506.05688/) (2025)
- [Multi-interaction TTS toward professional recording reproduction](https://mlsanity.com/paper/2507.00808/) (2025)
- [Multi-Label Annotation of Chest Abdomen Pelvis Computed Tomography Text Reports Using Deep Learning](https://mlsanity.com/paper/2102.02959/) (2021)
- [Leveraging Multi-domain, Heterogeneous Data using Deep Multitask Learning for Hate Speech Detection](https://mlsanity.com/paper/2103.12412/) (2021)
- [Lite Unified Modeling for Discriminative Reading Comprehension](https://mlsanity.com/paper/2203.14103/) (2022)
- [N-ACT: An Interpretable Deep Learning Model for Automatic Cell Type and Salient Gene Identification](https://mlsanity.com/paper/2206.04047/) (2022)
- [Investigation for Relative Voice Impression Estimation](https://mlsanity.com/paper/2602.14172/) (2026)
- [BCFNet: A Balanced Collaborative Filtering Network with Attention Mechanism](https://mlsanity.com/paper/2103.06105/) (2021)
- [Clustering Text Using Attention](https://mlsanity.com/paper/2201.02816/) (2022)
- [Assessment of few-hits machine learning classification algorithms for low energy physics in liquid argon detectors](https://mlsanity.com/paper/2305.09744/) (2023)
- [Enhancing Instance-Level Image Classification with Set-Level Labels](https://mlsanity.com/paper/2311.05659/) (2023)
- [Enhancing Apparent Personality Trait Analysis with Cross-Modal Embeddings](https://mlsanity.com/paper/2405.03846/) (2024)
- [Quantity versus Diversity: Influence of Data on Detecting EEG Pathology with Advanced ML Models](https://mlsanity.com/paper/2411.17709/) (2024)
- [Extracting Problem and Method Sentence from Scientific Papers: A Context-enhanced Transformer Using Formulaic Expression Desensitization](https://mlsanity.com/paper/2606.26481/) (2026)
- [Leveraging a Foundation Model for the EEG-Based Diagnosis of Alzheimer's Disease](https://mlsanity.com/paper/2608.27719/) (2026)
- [Multi-Step Forecasting of Grape Berry Temperature based on LSTM Model with Feed-Forward Attention](https://mlsanity.com/paper/2608.29008/) (2026)

## Abstract (arXiv)

We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.