mlsanityData to 7 Oct 2026

MC-LSTM: Mass-Conserving LSTM

arXiv 2101.05186 ↗First posted 13 Jan 2021cs.LG stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

The success of Convolutional Neural Networks (CNNs) in computer vision is mainly driven by their strong inductive bias, which is strong enough to allow CNNs to solve vision-related tasks with random weights, meaning without learning. Similarly, Long Short-Term Memory (LSTM) has a strong inductive bias towards storing information over time. However, many real-world systems are governed by conservation laws, which lead to the redistribution of particular quantities -- e.g. in physical and economical systems. Our novel Mass-Conserving LSTM (MC-LSTM) adheres to these conservation laws by extending the inductive bias of LSTM to model the redistribution of those stored quantities. MC-LSTMs set a new state-of-the-art for neural arithmetic units at learning arithmetic operations, such as addition tasks, which have a strong conservation law, as the sum is constant over time. Further, MC-LSTM is applied to traffic forecasting, modelling a pendulum, and a large benchmark dataset in hydrology, where it sets a new state-of-the-art for predicting peak flows. In the hydrology example, we show that MC-LSTM states correlate with real-world processes and are therefore interpretable.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 42 of its 104 references that are in the map. First posted on arXiv 13 Jan 2021.

Builds on

42 of 104 references.

References · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Layer Normalizationcited by 5,579
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  8. 2015Variational Inference with Normalizing Flowscited by 2,161
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  10. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  11. 2017Deep Image Priorcited by 1,097
  12. 2016HyperNetworkscited by 1,003
30 more
  1. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  2. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  3. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  4. 2019Hamiltonian Neural Networkscited by 702
  5. 2017Self-Normalizing Neural Networkscited by 591
  6. 2020Lagrangian Neural Networkscited by 388
  7. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  8. 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 335
  9. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  10. 2016Theory-guided Data Science: A New Paradigm for Scientific Discovery from Datacited by 169
  11. 2019Symplectic Recurrent Neural Networkscited by 148
  12. 2018Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecastingcited by 145
  13. 2018Discovering physical concepts with neural networkscited by 117
  14. 2016Full-Capacity Unitary Recurrent Neural Networkscited by 90
  15. 2017Orthogonal Recurrent Neural Networks with Scaled Cayley Transformcited by 82
  16. 2019Weight Agnostic Neural Networkscited by 71
  17. 2018Neural Arithmetic Logic Unitscited by 69
  18. 2019Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasetscited by 68
  19. 2016Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNscited by 62
  20. 2018Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profilescited by 55
  21. 2016Inductive Bias of Deep Convolutional Networks through Pooling Geometrycited by 53
  22. 2019Achieving Conservation of Energy in Neural Network Emulators for Climate Modelingcited by 27
  23. 2020Neural Arithmetic Unitscited by 25
  24. 2017Turing at SemEval-2017 Task 8: Sequential Approach to Rumour Stance Classification with Branch-LSTMcited by 20
  25. 2018Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodingscited by 18
  26. 2016A Powerful Generative Model Using Random Weights for the Deep Image Representationcited by 14
  27. 2019NeuralHydrology -- Interpreting LSTMs in Hydrologycited by 12
  28. 2020Neural Power Unitscited by 7
  29. 2018Markov Chain Neural Networkscited by 6
  30. 2020Neural Status Registerscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 2; 2024: 5; 2025: 3; 2026: 2.

17 papers.

Sort:

By area:

  1. 2021Machine learning accelerated computational fluid dynamicscited by 461
  2. 2023Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Modelcited by 16
  3. 2024Geometry-Informed Neural Networkscited by 12
  4. 2023Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrologycited by 11
  5. 2022A single Long Short-Term Memory network for enhancing the prediction of path-dependent plasticity with material heterogeneity and anisotropycited by 4
  6. 2022Learning Trajectories of Hamiltonian Systems with Neural Networkscited by 4
  7. 2022Accelerating hydrodynamic simulations of urban drainage systems with physics-guided machine learningcited by 4
  8. 2024Towards Interpretable Physical-Conceptual Catchment-Scale Hydrological Modeling using the Mass-Conserving-Perceptroncited by 4
  9. 2024Using Machine Learning to Discover Parsimonious and Physically-Interpretable Representations of Catchment-Scale Rainfall-Runoff Dynamicscited by 3
  10. 2025A Deep State Space Model for Rainfall-Runoff Simulationscited by 3
  11. 2025HydroDiffusion: Diffusion-Based Probabilistic Streamflow Forecasting with a State Space Backbonecited by 1
  12. 2022Multiple View Performers for Shape Completioncited by 0
5 more
  1. 2024Toward Routing River Water in Land Surface Models with Recurrent Neural Networkscited by 0
  2. 2024Streamflow Prediction with Uncertainty Quantification for Water Management: A Constrained Reasoning and Learning Approachcited by 0
  3. 2025Physics-guided Emulators Reveal Resilience and Fragility under Operational Latencies and Outagescited by 0
  4. 2026Physics-Informed CNN-LSTM for Street-Scale Urban Flood Prediction: Reconciling Aggregate Accuracy and Street-Level Plausibilitycited by 0
  5. 2026Hapi: A Multivariable Land-Surface Transformer for Medium-Range Hydrological Forecasting at Continental Scalecited by 0