# Conditional Neural Relational Inference for Interacting Systems

arXiv: https://arxiv.org/abs/2106.11083 · HTML: https://mlsanity.com/paper/2106.11083/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [A Compositional Object-Based Approach to Learning Physical Dynamics](https://mlsanity.com/paper/1612.00341/) (2016)
- [Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions](https://mlsanity.com/paper/1802.10353/) (2018)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [The Pose Knows: Video Forecasting by Generating Pose Futures](https://mlsanity.com/paper/1705.00053/) (2017)
- [A Neural Temporal Model for Human Motion Prediction](https://mlsanity.com/paper/1809.03036/) (2018)
- [VAIN: Attentional Multi-agent Predictive Modeling](https://mlsanity.com/paper/1706.06122/) (2017)
- [Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Prediction](https://mlsanity.com/paper/1910.02212/) (2019)
- [Imitation Learning for Human Pose Prediction](https://mlsanity.com/paper/1909.03449/) (2019)
- [Factorised Neural Relational Inference for Multi-Interaction Systems](https://mlsanity.com/paper/1905.08721/) (2019)
- [Permutation-equivariant neural networks applied to dynamics prediction](https://mlsanity.com/paper/1612.04530/) (2016)

## Built on by (most cited first, 28 of 28)

- [Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature](https://mlsanity.com/paper/2211.15779/) (2022)
- [Everything is Connected: Graph Neural Networks](https://mlsanity.com/paper/2301.08210/) (2023)
- [Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing](https://mlsanity.com/paper/2302.01524/) (2023)
- [Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observations](https://mlsanity.com/paper/2205.13479/) (2022)
- [Discrete-Valued Neural Communication](https://mlsanity.com/paper/2107.02367/) (2021)
- [Copula Conformal Prediction for Multi-step Time Series Forecasting](https://mlsanity.com/paper/2212.03281/) (2022)
- [Roto-translated Local Coordinate Frames For Interacting Dynamical Systems](https://mlsanity.com/paper/2110.14961/) (2021)
- [Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency](https://mlsanity.com/paper/2406.07598/) (2024)
- [Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models](https://mlsanity.com/paper/2209.04862/) (2022)
- [Using Multiple Vector Channels Improves E(n)-Equivariant Graph Neural Networks](https://mlsanity.com/paper/2309.03139/) (2023)
- [Signed Graph Neural Ordinary Differential Equation for Modeling Continuous-time Dynamics](https://mlsanity.com/paper/2312.11198/) (2023)
- [Disentangled Neural Relational Inference for Interpretable Motion Prediction](https://mlsanity.com/paper/2401.03599/) (2024)
- [Neural Message Passing for Visual Relationship Detection](https://mlsanity.com/paper/2208.04165/) (2022)
- [TANGO: Time-Reversal Latent GraphODE for Multi-Agent Dynamical Systems](https://mlsanity.com/paper/2310.06427/) (2023)
- [From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery](https://mlsanity.com/paper/2607.12474/) (2026)
- [Learning to Drop Out: An Adversarial Approach to Training Sequence VAEs](https://mlsanity.com/paper/2209.12590/) (2022)
- [Learning Physics-Guided Residual Dynamics for Deformable Object Simulation](https://mlsanity.com/paper/2607.13451/) (2026)
- [Exploiting Individual Graph Structures to Enhance Ecological Momentary Assessment (EMA) Forecasting](https://mlsanity.com/paper/2403.19442/) (2024)
- [F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data](https://mlsanity.com/paper/2406.16221/) (2024)
- [Adaptive Learning on User Segmentation: Universal to Specific Representation via Bipartite Neural Interaction](https://mlsanity.com/paper/2409.14945/) (2024)
- [EqCollide: Equivariant and Collision-Aware Deformable Objects Neural Simulator](https://mlsanity.com/paper/2506.05797/) (2025)
- [A Joint Spectro-Temporal Relational Thinking Based Acoustic Modeling Framework](https://mlsanity.com/paper/2409.15357/) (2024)
- [Learning Disease-Sensitive Latent Interaction Graphs From Noisy Cardiac Flow Measurements](https://mlsanity.com/paper/2602.23035/) (2026)
- [Subjective-Graph LLM Agents for Simulating Uncertainty in Classroom Social Perception](https://mlsanity.com/paper/2603.20750/) (2026)
- [BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Models](https://mlsanity.com/paper/2606.16489/) (2026)
- [A foundation model of numerical intelligence with cross-disciplinary generalization](https://mlsanity.com/paper/2607.28432/) (2026)
- [Fundamental Dynamical Units for Physics-Informed Structural Inference from Perturbation Time-Series in Networked Systems](https://mlsanity.com/paper/2609.11934/) (2026)
- [Structural Inference under Hidden Agents](https://mlsanity.com/paper/2609.18045/) (2026)

## Abstract (arXiv)

In this work, we want to learn to model the dynamics of similar yet distinct groups of interacting objects. These groups follow some common physical laws that exhibit specificities that are captured through some vectorial description. We develop a model that allows us to do conditional generation from any such group given its vectorial description. Unlike previous work on learning dynamical systems that can only do trajectory completion and require a part of the trajectory dynamics to be provided as input in generation time, we do generation using only the conditioning vector with no access to generation time's trajectories. We evaluate our model in the setting of modeling human gait and, in particular pathological human gait.