# LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1606.07461 · HTML: https://mlsanity.com/paper/1606.07461/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2016.

## Built on by (most cited first, 6 of 6)

- [CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models](https://mlsanity.com/paper/2109.01401/) (2021)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks](https://mlsanity.com/paper/2102.01792/) (2021)
- [Attention cannot be an Explanation](https://mlsanity.com/paper/2201.11194/) (2022)
- [State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions](https://mlsanity.com/paper/2212.05178/) (2022)
- [Learning to Remember: Distilling Memory Retention for Compact Recurrent Neural Networks](https://mlsanity.com/paper/2610.06942/) (2026)

## Abstract (arXiv)

Recurrent neural networks, and in particular long short-term memory (LSTM) networks, are a remarkably effective tool for sequence modeling that learn a dense black-box hidden representation of their sequential input. Researchers interested in better understanding these models have studied the changes in hidden state representations over time and noticed some interpretable patterns but also significant noise. In this work, we present LSTMVIS, a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics. The tool allows users to select a hypothesis input range to focus on local state changes, to match these states changes to similar patterns in a large data set, and to align these results with structural annotations from their domain. We show several use cases of the tool for analyzing specific hidden state properties on dataset containing nesting, phrase structure, and chord progressions, and demonstrate how the tool can be used to isolate patterns for further statistical analysis. We characterize the domain, the different stakeholders, and their goals and tasks.