mlsanityData to 7 Oct 2026

Long Short-Term Memory-Networks for Machine Reading

arXiv 1601.06733 ↗First posted 25 Jan 2016cs.CL cs.NEcited by 161

Not on the map. Why?

Abstract (arXiv)

In this paper we address the question of how to render sequence-level networks better at handling structured input. We propose a machine reading simulator which processes text incrementally from left to right and performs shallow reasoning with memory and attention. The reader extends the Long Short-Term Memory architecture with a memory network in place of a single memory cell. This enables adaptive memory usage during recurrence with neural attention, offering a way to weakly induce relations among tokens. The system is initially designed to process a single sequence but we also demonstrate how to integrate it with an encoder-decoder architecture. Experiments on language modeling, sentiment analysis, and natural language inference show that our model matches or outperforms the state of the art.

Key facts. Cited by 161 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jan 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 0; 2019: 0; 2020: 0; 2021: 53; 2022: 41; 2023: 28; 2024: 22; 2025: 13; 2026: 3.

161 papers.

Sort:

By area:

  1. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  2. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  3. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  4. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
  5. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  6. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 149
  7. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  8. 2021Do Transformer Modifications Transfer Across Implementations and Applications?cited by 87
  9. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  10. 2023Transformers as Support Vector Machinescited by 67
  11. 2022CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentationcited by 60
  12. 2022Improving Multi-turn Emotional Support Dialogue Generation with Lookahead Strategy Planningcited by 59
38 more
  1. 2023Max-Margin Token Selection in Attention Mechanismcited by 57
  2. 2022Swin Transformer for Fast MRIcited by 56
  3. 2024Toward Visual Grounding: A Surveycited by 52
  4. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  5. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  6. 2022Simplicial Attention Networkscited by 40
  7. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  8. 2023On the Optimization and Generalization of Multi-head Attentioncited by 37
  9. 2021Explain and Predict, and then Predict Againcited by 29
  10. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  11. 2023Generalized Simplicial Attention Neural Networkscited by 27
  12. 2023Approximating Two-Layer Feedforward Networks for Efficient Transformerscited by 27
  13. 2021Learning Conjoint Attentions for Graph Neural Netscited by 26
  14. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  15. 2021Bayesian Transformer Language Models for Speech Recognitioncited by 22
  16. 2017Reinforced Video Captioning with Entailment Rewardscited by 21
  17. 2022Simplicial Attention Neural Networkscited by 21
  18. 2025Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuningcited by 21
  19. 2022Survey on Automated Short Answer Grading with Deep Learning: from Word Embeddings to Transformerscited by 20
  20. 2021Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causalitycited by 17
  21. 2022Astronomia ex machina: a history, primer, and outlook on neural networks in astronomycited by 17
  22. 2022CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Drivingcited by 16
  23. 2021CLVSA: A Convolutional LSTM Based Variational Sequence-to-Sequence Model with Attention for Predicting Trends of Financial Marketscited by 15
  24. 2024A Comprehensive Survey of Compression Algorithms for Language Modelscited by 15
  25. 2021AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsingcited by 14
  26. 2022S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attentioncited by 14
  27. 2022Paraphrase Identification with Deep Learning: A Review of Datasets and Methodscited by 13
  28. 2021Miti-DETR: Object Detection based on Transformers with Mitigatory Self-Attention Convergencecited by 12
  29. 2022SwinCheX: Multi-label classification on chest X-ray images with transformerscited by 12
  30. 2022GAMR: A Guided Attention Model for (visual) Reasoningcited by 12
  31. 2022kMaX-DeepLab: k-means Mask Transformercited by 12
  32. 2023Is attention all you need in medical image analysis? A reviewcited by 12
  33. 2021CaEGCN: Cross-Attention Fusion based Enhanced Graph Convolutional Network for Clusteringcited by 11
  34. 2021Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learningcited by 11
  35. 2023Student-friendly Knowledge Distillationcited by 11
  36. 2024From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?cited by 11
  37. 2024Next-token prediction capacity: general upper bounds and a lower bound for transformerscited by 11
  38. 2021Mixed Precision of Quantization of Transformer Language Models for Speech Recognitioncited by 10

50 most cited of 161 All 161 (JSON)