# Cooperative data-driven modeling

arXiv: https://arxiv.org/abs/2211.12971 · HTML: https://mlsanity.com/paper/2211.12971/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 77 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Learning Combinatorial Optimization Algorithms over Graphs](https://mlsanity.com/paper/1704.01665/) (2017)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [What's Hidden in a Randomly Weighted Neural Network?](https://mlsanity.com/paper/1911.13299/) (2019)
- [Supermasks in Superposition](https://mlsanity.com/paper/2006.14769/) (2020)
- [Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures](https://mlsanity.com/paper/1607.03250/) (2016)
- [Continual Lifelong Learning in Natural Language Processing: A Survey](https://mlsanity.com/paper/2012.09823/) (2020)
- [Continual Learning via Neural Pruning](https://mlsanity.com/paper/1903.04476/) (2019)
- [SpaceNet: Make Free Space For Continual Learning](https://mlsanity.com/paper/2007.07617/) (2020)
- [Geometric deep learning for computational mechanics Part I: Anisotropic Hyperelasticity](https://mlsanity.com/paper/2001.04292/) (2020)
- [A deep material network for multiscale topology learning and accelerated nonlinear modeling of heterogeneous materials](https://mlsanity.com/paper/1807.09829/) (2018)
- [NN-EUCLID: deep-learning hyperelasticity without stress data](https://mlsanity.com/paper/2205.06664/) (2022)
- [The mixed deep energy method for resolving concentration features in finite strain hyperelasticity](https://mlsanity.com/paper/2104.09623/) (2021)
- [Continual Prune-and-Select: Class-incremental learning with specialized subnetworks](https://mlsanity.com/paper/2208.04952/) (2022)
- [On-the-fly construction of surrogate constitutive models for concurrent multiscale mechanical analysis through probabilistic machine learning](https://mlsanity.com/paper/2007.07749/) (2020)
- [On Class Orderings for Incremental Learning](https://mlsanity.com/paper/2007.02145/) (2020)
- [Neural network relief: a pruning algorithm based on neural activity](https://mlsanity.com/paper/2109.10795/) (2021)
- [Exploring the potential of transfer learning for metamodels of heterogeneous material deformation](https://mlsanity.com/paper/2010.16260/) (2020)

## Built on by (most cited first, 5 of 5)

- [Automatically Differentiable Model Updating (ADiMU): conventional, hybrid, and neural network material model discovery including history-dependency](https://mlsanity.com/paper/2505.07801/) (2025)
- [Recurrent neural networks and transfer learning for elasto-plasticity in woven composites](https://mlsanity.com/paper/2311.13434/) (2023)
- [COMMET: orders-of-magnitude speed-up in finite element method via batch-vectorized neural constitutive updates](https://mlsanity.com/paper/2510.00884/) (2025)
- [A machine learning-based viscoelastic-viscoplastic model for epoxy nanocomposites with moisture content](https://mlsanity.com/paper/2305.08102/) (2023)
- [Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling](https://mlsanity.com/paper/2507.13416/) (2025)

## Abstract (arXiv)

Data-driven modeling in mechanics is evolving rapidly based on recent machine learning advances, especially on artificial neural networks. As the field matures, new data and models created by different groups become available, opening possibilities for cooperative modeling. However, artificial neural networks suffer from catastrophic forgetting, i.e. they forget how to perform an old task when trained on a new one. This hinders cooperation because adapting an existing model for a new task affects the performance on a previous task trained by someone else. The authors developed a continual learning method that addresses this issue, applying it here for the first time to solid mechanics. In particular, the method is applied to recurrent neural networks to predict history-dependent plasticity behavior, although it can be used on any other architecture (feedforward, convolutional, etc.) and to predict other phenomena. This work intends to spawn future developments on continual learning that will foster cooperative strategies among the mechanics community to solve increasingly challenging problems. We show that the chosen continual learning strategy can sequentially learn several constitutive laws without forgetting them, using less data to achieve the same error as standard (non-cooperative) training of one law per model.