mlsanityData to 14 Oct 2026

Exposing the Functionalities of Neurons for Gated Recurrent Unit Based Sequence-to-Sequence Model

arXiv 2303.15072 ↗First posted 27 Mar 2023cs.NE cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The goal of this paper is to report certain scientific discoveries about a Seq2Seq model. It is known that analyzing the behavior of RNN-based models at the neuron level is considered a more challenging task than analyzing a DNN or CNN models due to their recursive mechanism in nature. This paper aims to provide neuron-level analysis to explain why a vanilla GRU-based Seq2Seq model without attention can achieve token-positioning. We found four different types of neurons: storing, counting, triggering, and outputting and further uncover the mechanism for these neurons to work together in order to produce the right token in the right position.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

27 of 42 references.

References · 27

  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  6. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  7. 2019Attention is not Explanationcited by 929
  8. 2014Memory Networkscited by 363
  9. 2016A Persona-Based Neural Conversation Modelcited by 223
  10. 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
  11. 2015Visualizing and Understanding Recurrent Networkscited by 160
  12. 2016Language to Logical Form with Neural Attentioncited by 158
15 more
  1. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 149
  2. 2017Learning to Generate Reviews and Discovering Sentimentcited by 148
  3. 2015Neural Responding Machine for Short-Text Conversationcited by 124
  4. 2018On the Practical Computational Power of Finite Precision RNNs for Language Recognitioncited by 117
  5. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  6. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  7. 2015Multi-task Sequence to Sequence Learningcited by 94
  8. 2019The emergence of number and syntax units in LSTM language modelscited by 94
  9. 2017Generating Sentences by Editing Prototypescited by 88
  10. 2016Controlling Output Length in Neural Encoder-Decoderscited by 75
  11. 2018Content preserving text generation with attribute controlscited by 47
  12. 2016Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Stringscited by 27
  13. 2013Complexity of Word Collocation Networks: A Preliminary Structural Analysiscited by 17
  14. 2015Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Rerankingcited by 17
  15. 2018Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarizationcited by 3

Built on by

Not cited yet.