Learning to Decouple Complex Systems
Not on the map. Why?
Abstract (arXiv)
A complex system with cluttered observations may be a coupled mixture of multiple simple sub-systems corresponding to latent entities. Such sub-systems may hold distinct dynamics in the continuous-time domain; therein, complicated interactions between sub-systems also evolve over time. This setting is fairly common in the real world but has been less considered. In this paper, we propose a sequential learning approach under this setting by decoupling a complex system for handling irregularly sampled and cluttered sequential observations. Such decoupling brings about not only subsystems describing the dynamics of each latent entity but also a meta-system capturing the interaction between entities over time. Specifically, we argue that the meta-system evolving within a simplex is governed by projected differential equations (ProjDEs). We further analyze and provide neural-friendly projection operators in the context of Bregman divergence. Experimental results on synthetic and real-world datasets show the advantages of our approach when facing complex and cluttered sequential data compared to the state-of-the-art.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 3 Feb 2023.
Builds on
44 of 71 references.
References · 44
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
- 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
- 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
- 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
- 2019Augmented Neural ODEscited by 396
- 2018Neural Relational Inference for Interacting Systemscited by 339
- 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 337
- 2020Scalable Gradients for Stochastic Differential Equationscited by 264
32 more
- 2021GRAND: Graph Neural Diffusioncited by 258
- 2017A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Predictioncited by 245
- 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
- 2019Recurrent Independent Mechanismscited by 219
- 2019Sparse Sequence-to-Sequence Modelscited by 167
- 2018Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNNcited by 142
- 2021Enhancing Computational Fluid Dynamics with Machine Learningcited by 137
- 2016An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Datacited by 126
- 2019Continuous Graph Neural Networkscited by 114
- 2020Learning Stable Deep Dynamics Modelscited by 110
- 2019Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of Viewcited by 108
- 2019Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noisecited by 98
- 2021Beltrami Flow and Neural Diffusion on Graphscited by 84
- 2020Neural Manifold Ordinary Differential Equationscited by 81
- 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
- 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
- 2020Learning continuous-time PDEs from sparse data with graph neural networkscited by 53
- 2017Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifiercited by 47
- 2020A Mathematical Theory of Attentioncited by 43
- 2019Graph Neural Networks for Modelling Traffic Participant Interactioncited by 42
- 2019Deep learning to discover and predict dynamics on an inertial manifoldcited by 42
- 2020Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equationcited by 39
- 2019Independence Promoted Graph Disentangled Networkscited by 38
- 2021Fast and Slow Learning of Recurrent Independent Mechanismscited by 36
- 2017Discrete Event, Continuous Time RNNscited by 31
- 2016A scalable end-to-end Gaussian process adapter for irregularly sampled time series classificationcited by 27
- 2021Neural Delay Differential Equationscited by 26
- 2019Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systemscited by 25
- 2021Fast Stochastic Bregman Gradient Methods: Sharp Analysis and Variance Reductioncited by 19
- 2020Complex Sequential Understanding through the Awareness of Spatial and Temporal Conceptscited by 8
- 2020Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Trainingcited by 7
- 2022Learning Interacting Dynamical Systems with Latent Gaussian Process ODEscited by 5
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.