mlsanityData to 7 Oct 2026

Visualizing and Understanding Recurrent Networks

arXiv 1506.02078 ↗First posted 5 Jun 2015cs.LG cs.CL cs.NEcited by 158

Not on the map. Why?

Abstract (arXiv)

Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data. However, while LSTMs provide exceptional results in practice, the source of their performance and their limitations remain rather poorly understood. Using character-level language models as an interpretable testbed, we aim to bridge this gap by providing an analysis of their representations, predictions and error types. In particular, our experiments reveal the existence of interpretable cells that keep track of long-range dependencies such as line lengths, quotes and brackets. Moreover, our comparative analysis with finite horizon n-gram models traces the source of the LSTM improvements to long-range structural dependencies. Finally, we provide analysis of the remaining errors and suggests areas for further study.

Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 44; 2022: 27; 2023: 28; 2024: 34; 2025: 18; 2026: 7.

158 papers.

Sort:

By area:

  1. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  2. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  3. 2024Are self-explanations from Large Language Models faithful?cited by 101
  4. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  5. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 88
  6. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  7. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 82
  8. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  9. 2021Explaining Time Series Predictions with Dynamic Maskscited by 72
  10. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  11. 2024A Multimodal Automated Interpretability Agentcited by 62
  12. 2023Explaining black box text modules in natural language with language modelscited by 61
38 more
  1. 2024Confidence Regulation Neurons in Language Modelscited by 57
  2. 2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
  3. 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
  4. 2022The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanationscited by 41
  5. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  6. 2022Model Zoos: A Dataset of Diverse Populations of Neural Network Modelscited by 38
  7. 2022Analyzing Encoded Concepts in Transformer Language Modelscited by 37
  8. 2022On the Explainability of Natural Language Processing Deep Modelscited by 37
  9. 2023Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stabilitycited by 34
  10. 2023A Preliminary Study of the Intrinsic Relationship between Complexity and Alignmentcited by 33
  11. 2021Controlling Conditional Language Models without Catastrophic Forgettingcited by 32
  12. 2024An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixelscited by 32
  13. 2021Improving Deep Learning for HAR with shallow LSTMscited by 30
  14. 2023WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognitioncited by 30
  15. 2021Explain and Predict, and then Predict Againcited by 29
  16. 2021M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysiscited by 29
  17. 2021Towards More Fine-grained and Reliable NLP Performance Predictioncited by 26
  18. 2023Visual Analytics For Machine Learning: A Data Perspective Surveycited by 25
  19. 2024Mechanistic?cited by 25
  20. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  21. 2024Configurable Foundation Models: Building LLMs from a Modular Perspectivecited by 22
  22. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21
  23. 2021Discovering Distinctive "Semantics" in Super-Resolution Networkscited by 20
  24. 2021CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Modelscited by 20
  25. 2024From Insights to Actions: The Impact of Interpretability and Analysis Research on NLPcited by 20
  26. 2021VisQA: X-raying Vision and Language Reasoning in Transformerscited by 19
  27. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  28. 2023Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?cited by 19
  29. 2022A Unified Understanding of Deep NLP Models for Text Classificationcited by 18
  30. 2022Extracting Finite Automata from RNNs Using State Mergingcited by 17
  31. 2022Explainability in Graph Neural Networks: An Experimental Surveycited by 16
  32. 2023Can LLMs facilitate interpretation of pre-trained language models?cited by 15
  33. 2021A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networkscited by 14
  34. 2021Mediators in Determining what Processing BERT Performs Firstcited by 14
  35. 2025Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scalecited by 13
  36. 2021An Operator Theoretic Approach for Analyzing Sequence Neural Networkscited by 12
  37. 2021Security Vulnerability Detection Using Deep Learning Natural Language Processingcited by 12
  38. 2021Distinguishing rule- and exemplar-based generalization in learning systemscited by 12

50 most cited of 158 All 158 (JSON)