mlsanityData to 7 Oct 2026

NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks

arXiv 1812.09359 ↗First posted 21 Dec 2018cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

We present a toolkit to facilitate the interpretation and understanding of neural network models. The toolkit provides several methods to identify salient neurons with respect to the model itself or an external task. A user can visualize selected neurons, ablate them to measure their effect on the model accuracy, and manipulate them to control the behavior of the model at the test time. Such an analysis has a potential to serve as a springboard in various research directions, such as understanding the model, better architectural choices, model distillation and controlling data biases.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 10; 2023: 7; 2024: 5; 2025: 2; 2026: 1.

31 papers.

Sort:

By area:

  1. 2022Discovering Latent Concepts Learned in BERTcited by 56
  2. 2022Analyzing Encoded Concepts in Transformer Language Modelscited by 37
  3. 2021How transfer learning impacts linguistic knowledge in deep NLP models?cited by 32
  4. 2022LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Modelscited by 28
  5. 2024Large Language Models Relearn Removed Conceptscited by 26
  6. 2022On the Transformation of Latent Space in Fine-Tuned NLP Modelscited by 19
  7. 2021LMdiff: A Visual Diff Tool to Compare Language Modelscited by 17
  8. 2022Probing for Constituency Structure in Neural Language Modelscited by 13
  9. 2024Exploring Alignment in Shared Cross-lingual Spacescited by 13
  10. 2023NeuroX Library for Neuron Analysis of Deep NLP Modelscited by 11
  11. 2021What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysiscited by 9
  12. 2023The Role of Interactive Visualization in Explaining (Large) NLP Models: from Data to Inferencecited by 9
19 more
  1. 2023Evaluating Neuron Interpretation Methods of NLP Modelscited by 8
  2. 2023Examining the Causal Effect of First Names on Language Models: The Case of Social Commonsense Reasoningcited by 8
  3. 2022ConceptX: A Framework for Latent Concept Analysiscited by 6
  4. 2025Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attributioncited by 5
  5. 2021Fine-grained Interpretation and Causation Analysis in Deep NLP Modelscited by 4
  6. 2022An Interpretability Evaluation Benchmark for Pre-trained Language Modelscited by 4
  7. 2021Detection Accuracy for Evaluating Compositional Explanations of Unitscited by 3
  8. 2022Universal and Independent: Multilingual Probing Framework for Exhaustive Model Interpretation and Evaluationcited by 3
  9. 2023Interpretability in Activation Space Analysis of Transformers: A Focused Surveycited by 3
  10. 2024Detecting Conceptual Abstraction in LLMscited by 3
  11. 2025Evaluating Sparse Autoencoders for Monosemantic Representationcited by 3
  12. 2021NEWRON: A New Generalization of the Artificial Neuron to Enhance the Interpretability of Neural Networkscited by 1
  13. 2022Post-hoc analysis of Arabic transformer modelscited by 1
  14. 2023NxPlain: Web-based Tool for Discovery of Latent Conceptscited by 1
  15. 2023Redundancy and Concept Analysis for Code-trained Language Modelscited by 1
  16. 2024Consistency Matters: Explore LLMs Consistency From a Black-Box Perspectivecited by 1
  17. 2026From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Modelscited by 1
  18. 2022Interpreting Arabic Transformer Modelscited by 0
  19. 2024Linking in Style: Understanding learned features in deep learning modelscited by 0