mlsanityData to 14 Oct 2026

Interpretable Textual Neuron Representations for NLP

arXiv 1809.07291 ↗First posted 19 Sep 2018cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

Input optimization methods, such as Google Deep Dream, create interpretable representations of neurons for computer vision DNNs. We propose and evaluate ways of transferring this technology to NLP. Our results suggest that gradient ascent with a gumbel softmax layer produces n-gram representations that outperform naive corpus search in terms of target neuron activation. The representations highlight differences in syntax awareness between the language and visual models of the Imaginet architecture.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 4; 2023: 7; 2024: 1; 2025: 1.

16 papers.

Sort:

By area:

  1. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  2. 2021An Interpretability Illusion for BERTcited by 97
  3. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  4. 2023Explaining black box text modules in natural language with language modelscited by 62
  5. 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 27
  6. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21
  7. 2022Towards Explainable Evaluation Metrics for Natural Language Generationcited by 15
  8. 2023Evaluating Neuron Interpretation Methods of NLP Modelscited by 8
  9. 2023Examining the Causal Effect of First Names on Language Models: The Case of Social Commonsense Reasoningcited by 8
  10. 2024Fluent dreaming for language modelscited by 5
  11. 2021Fine-grained Interpretation and Causation Analysis in Deep NLP Modelscited by 4
  12. 2023N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Modelscited by 4
4 more
  1. 2023Investigating the Encoding of Words in BERT's Neurons using Feature Textualizationcited by 3
  2. 2022Explaining Any ML Model? -- On Goals and Capabilities of XAIcited by 1
  3. 2023NxPlain: Web-based Tool for Discovery of Latent Conceptscited by 1
  4. 2025ContextBench: Modifying Contexts for Targeted Latent Activationcited by 0