mlsanityData to 7 Oct 2026

On Multiplicative Integration with Recurrent Neural Networks

arXiv 1606.06630 ↗First posted 21 Jun 2016cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

We introduce a general and simple structural design called Multiplicative Integration (MI) to improve recurrent neural networks (RNNs). MI changes the way in which information from difference sources flows and is integrated in the computational building block of an RNN, while introducing almost no extra parameters. The new structure can be easily embedded into many popular RNN models, including LSTMs and GRUs. We empirically analyze its learning behaviour and conduct evaluations on several tasks using different RNN models. Our experimental results demonstrate that Multiplicative Integration can provide a substantial performance boost over many of the existing RNN models.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 1; 2023: 0; 2024: 3; 2025: 2; 2026: 4.

15 papers.

Sort:

  1. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  2. 2022High-Order Pooling for Graph Neural Networks with Tensor Decompositioncited by 24
  3. 2021Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Unitscited by 8
  4. 2021Explainable Natural Language Processing with Matrix Product Statescited by 7
  5. 2021On the Memory Mechanism of Tensor-Power Recurrent Modelscited by 5
  6. 2025Revisiting Bi-Linear State Transitions in Recurrent Neural Networkscited by 4
  7. 2024Stability Analysis of Various Symbolic Rule Extraction Methods from Recurrent Neural Networkcited by 3
  8. 2024Language Modeling Using Tensor Trainscited by 3
  9. 2024A Tensor Decomposition Perspective on Second-order RNNscited by 3
  10. 2025Spiking Manifestocited by 3
  11. 2021Gates Are Not What You Need in RNNscited by 1
  12. 2026Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learningcited by 1
3 more
  1. 2026Modeling Next-Token Prediction as Left-Nested Intuitionistic Implicationcited by 0
  2. 2026On the Role of Depth in the Expressivity of RNNscited by 0
  3. 2026Gating Enables Curvature: A Geometric Expressivity Gap in Attentioncited by 0