mlsanityData to 7 Oct 2026

Identifying and Controlling Important Neurons in Neural Machine Translation

arXiv 1811.01157 ↗First posted 3 Nov 2018cs.CLcited by 110

Not on the map. Why?

Abstract (arXiv)

Neural machine translation (NMT) models learn representations containing substantial linguistic information. However, it is not clear if such information is fully distributed or if some of it can be attributed to individual neurons. We develop unsupervised methods for discovering important neurons in NMT models. Our methods rely on the intuition that different models learn similar properties, and do not require any costly external supervision. We show experimentally that translation quality depends on the discovered neurons, and find that many of them capture common linguistic phenomena. Finally, we show how to control NMT translations in predictable ways, by modifying activations of individual neurons.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 16; 2023: 24; 2024: 24; 2025: 13; 2026: 12.

110 papers.

Sort:

By area:

  1. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  2. 2023Explainability for Large Language Models: A Surveycited by 331
  3. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  4. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  5. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  6. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  7. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  8. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  9. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  10. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 124
  11. 2024Universal Neurons in GPT2 Language Modelscited by 100
  12. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 92
38 more
  1. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  2. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  3. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  4. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  5. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
  6. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  7. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 61
  8. 2022Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictionscited by 57
  9. 2022Discovering Latent Concepts Learned in BERTcited by 56
  10. 2021PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognitioncited by 50
  11. 2021NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternativescited by 46
  12. 2023Break It Down: Evidence for Structural Compositionality in Neural Networkscited by 44
  13. 2021The Low-Dimensional Linear Geometry of Contextualized Word Representationscited by 39
  14. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  15. 2022Analyzing Encoded Concepts in Transformer Language Modelscited by 37
  16. 2021The Grammar-Learning Trajectories of Neural Language Modelscited by 36
  17. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  18. 2023A Preliminary Study of the Intrinsic Relationship between Complexity and Alignmentcited by 33
  19. 2021How transfer learning impacts linguistic knowledge in deep NLP models?cited by 32
  20. 2024Towards Universality: Studying Mechanistic Similarity Across Language Model Architecturescited by 31
  21. 2023How Does Attention Work in Vision Transformers? A Visual Analytics Attemptcited by 29
  22. 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 26
  23. 2024Large Language Models Relearn Removed Conceptscited by 26
  24. 2024Holmes: A Benchmark to Assess the Linguistic Competence of Language Modelscited by 26
  25. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26
  26. 2022Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Valuescited by 25
  27. 2024Mechanistic?cited by 25
  28. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  29. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  30. 2023Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Modelcited by 22
  31. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 22
  32. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21
  33. 2024Constructing Benchmarks and Interventions for Combating Hallucinations in LLMscited by 19
  34. 2021GAM Changer: Editing Generalized Additive Models with Interactive Visualizationcited by 18
  35. 2025Robustly Improving LLM Fairness in Realistic Settings via Interpretabilitycited by 16
  36. 2021Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translationcited by 15
  37. 2023Can LLMs facilitate interpretation of pre-trained language models?cited by 15
  38. 2024Vision-Language Models Create Cross-Modal Task Representationscited by 14

50 most cited of 110 All 110 (JSON)