# MC-LSTM: Mass-Conserving LSTM

arXiv: https://arxiv.org/abs/2101.05186 · HTML: https://mlsanity.com/paper/2101.05186/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 104 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting](https://mlsanity.com/paper/1802.07007/) (2018)
- [Discovering physical concepts with neural networks](https://mlsanity.com/paper/1807.10300/) (2018)
- [Full-Capacity Unitary Recurrent Neural Networks](https://mlsanity.com/paper/1611.00035/) (2016)
- [Orthogonal Recurrent Neural Networks with Scaled Cayley Transform](https://mlsanity.com/paper/1707.09520/) (2017)
- [Weight Agnostic Neural Networks](https://mlsanity.com/paper/1906.04358/) (2019)
- [Neural Arithmetic Logic Units](https://mlsanity.com/paper/1808.00508/) (2018)
- [Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets](https://mlsanity.com/paper/1907.08456/) (2019)

## Built on by (most cited first, 17 of 17)

- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Model](https://mlsanity.com/paper/2312.08403/) (2023)
- [Geometry-Informed Neural Networks](https://mlsanity.com/paper/2402.14009/) (2024)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Towards Interpretable Physical-Conceptual Catchment-Scale Hydrological Modeling using the Mass-Conserving-Perceptron](https://mlsanity.com/paper/2401.14521/) (2024)
- [A single Long Short-Term Memory network for enhancing the prediction of path-dependent plasticity with material heterogeneity and anisotropy](https://mlsanity.com/paper/2204.01466/) (2022)
- [Learning Trajectories of Hamiltonian Systems with Neural Networks](https://mlsanity.com/paper/2204.05077/) (2022)
- [Accelerating hydrodynamic simulations of urban drainage systems with physics-guided machine learning](https://mlsanity.com/paper/2206.01538/) (2022)
- [Using Machine Learning to Discover Parsimonious and Physically-Interpretable Representations of Catchment-Scale Rainfall-Runoff Dynamics](https://mlsanity.com/paper/2412.04845/) (2024)
- [A Deep State Space Model for Rainfall-Runoff Simulations](https://mlsanity.com/paper/2501.14980/) (2025)
- [HydroDiffusion: Diffusion-Based Probabilistic Streamflow Forecasting with a State Space Backbone](https://mlsanity.com/paper/2512.12183/) (2025)
- [Multiple View Performers for Shape Completion](https://mlsanity.com/paper/2209.06291/) (2022)
- [Toward Routing River Water in Land Surface Models with Recurrent Neural Networks](https://mlsanity.com/paper/2404.14212/) (2024)
- [Streamflow Prediction with Uncertainty Quantification for Water Management: A Constrained Reasoning and Learning Approach](https://mlsanity.com/paper/2406.00133/) (2024)
- [Physics-guided Emulators Reveal Resilience and Fragility under Operational Latencies and Outages](https://mlsanity.com/paper/2510.18535/) (2025)
- [Physics-Informed CNN-LSTM for Street-Scale Urban Flood Prediction: Reconciling Aggregate Accuracy and Street-Level Plausibility](https://mlsanity.com/paper/2607.25148/) (2026)
- [Hapi: A Multivariable Land-Surface Transformer for Medium-Range Hydrological Forecasting at Continental Scale](https://mlsanity.com/paper/2609.22702/) (2026)

## Abstract (arXiv)

The success of Convolutional Neural Networks (CNNs) in computer vision is mainly driven by their strong inductive bias, which is strong enough to allow CNNs to solve vision-related tasks with random weights, meaning without learning. Similarly, Long Short-Term Memory (LSTM) has a strong inductive bias towards storing information over time. However, many real-world systems are governed by conservation laws, which lead to the redistribution of particular quantities -- e.g. in physical and economical systems. Our novel Mass-Conserving LSTM (MC-LSTM) adheres to these conservation laws by extending the inductive bias of LSTM to model the redistribution of those stored quantities. MC-LSTMs set a new state-of-the-art for neural arithmetic units at learning arithmetic operations, such as addition tasks, which have a strong conservation law, as the sum is constant over time. Further, MC-LSTM is applied to traffic forecasting, modelling a pendulum, and a large benchmark dataset in hydrology, where it sets a new state-of-the-art for predicting peak flows. In the hydrology example, we show that MC-LSTM states correlate with real-world processes and are therefore interpretable.