mlsanityData to 7 Oct 2026

Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement

arXiv 2212.05970 ↗First posted 9 Dec 2022cs.SE cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Can we take a recurrent neural network (RNN) trained to translate between languages and augment it to support a new natural language without retraining the model from scratch? Can we fix the faulty behavior of the RNN by replacing portions associated with the faulty behavior? Recent works on decomposing a fully connected neural network (FCNN) and convolutional neural network (CNN) into modules have shown the value of engineering deep models in this manner, which is standard in traditional SE but foreign for deep learning models. However, prior works focus on the image-based multiclass classification problems and cannot be applied to RNN due to (a) different layer structures, (b) loop structures, (c) different types of input-output architectures, and (d) usage of both nonlinear and logistic activation functions. In this work, we propose the first approach to decompose an RNN into modules. We study different types of RNNs, i.e., Vanilla, LSTM, and GRU. Further, we show how such RNN modules can be reused and replaced in various scenarios. We evaluate our approach against 5 canonical datasets (i.e., Math QA, Brown Corpus, Wiki-toxicity, Clinc OOS, and Tatoeba) and 4 model variants for each dataset. We found that decomposing a trained model has a small cost (Accuracy: -0.6%, BLEU score: +0.10%). Also, the decomposed modules can be reused and replaced without needing to retrain.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 47 references that are in the map. First posted on arXiv 9 Dec 2022.

Builds on

18 of 47 references.

References · 18

  1. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  2. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  3. 2017Dynamic Routing Between Capsulescited by 689
  4. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  5. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  6. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  7. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  8. 2015Neural Module Networkscited by 375
  9. 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 223
  10. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  11. 2017Learning to Reason: End-to-End Module Networks for Visual Question Answeringcited by 125
  12. 2017Recent Advances in Recurrent Neural Networkscited by 89
6 more
  1. 2021Traceability Transformed: Generating more Accurate Links with Pre-Trained BERT Modelscited by 20
  2. 2021Decomposing Convolutional Neural Networks into Reusable and Replaceable Modulescited by 14
  3. 2019Recursive Sketches for Modular Deep Learningcited by 11
  4. 2019AdapNet: Adaptability Decomposing Encoder-Decoder Network for Weakly Supervised Action Recognition and Localizationcited by 8
  5. 2021Automating the Removal of Obsolete TODO Commentscited by 6
  6. 2018HSD-CNN: Hierarchically self decomposing CNN architecture using class specific filter sensitivity analysiscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025Scatter-Based Innovation Propagation in Large Language Models for Multi-Stage Process Adaptationcited by 2
  2. 2024Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deploymentcited by 1
  3. 2025NeMo: A Neuron-Level Modularizing-While-Training Approach for Decomposing DNN Modelscited by 0