# Deep Efficient Continuous Manifold Learning for Time Series Modeling

arXiv: https://arxiv.org/abs/2112.03379 · HTML: https://mlsanity.com/paper/2112.03379/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 87 references that are in the map.
- First posted on arXiv 3 Dec 2021.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [Benchmarking Multivariate Time Series Classification Algorithms](https://mlsanity.com/paper/2007.13156/) (2020)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [Continuous Graph Neural Networks](https://mlsanity.com/paper/1912.00967/) (2019)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging](https://mlsanity.com/paper/1910.11162/) (2019)
- [XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging](https://mlsanity.com/paper/2007.05492/) (2020)

## Built on by (most cited first, 10 of 10)

- [Deep Geometric Learning with Monotonicity Constraints for Alzheimer's Disease Progression](https://mlsanity.com/paper/2310.03353/) (2023)
- [MRI Contrast Enhancement Kinetics World Model](https://mlsanity.com/paper/2602.19285/) (2026)
- [Geometry of orofacial neuromuscular signals: speech articulation decoding using surface electromyography](https://mlsanity.com/paper/2411.02591/) (2024)
- [GeoDynamics: A Geometric State-Space Neural Network for Understanding Brain Dynamics on Riemannian Manifolds](https://mlsanity.com/paper/2601.13570/) (2026)
- [Riemannian Geometry for the classification of brain states with intracortical brain-computer interfaces](https://mlsanity.com/paper/2504.05534/) (2025)
- [Alzheimers Disease Progression Prediction Based on Manifold Mapping of Irregularly Sampled Longitudinal Data](https://mlsanity.com/paper/2511.20154/) (2025)
- [Multivariate Time Series Forecasting with Hybrid Euclidean-SPD Manifold Graph Neural Networks](https://mlsanity.com/paper/2512.14023/) (2025)
- [Intrinsic-Metric Physics-Informed Neural Networks (IM-PINN) for Reaction-Diffusion Dynamics on Complex Riemannian Manifolds](https://mlsanity.com/paper/2601.00834/) (2025)
- [Permutation-Equivariant 2D State Space Models: Theory and Canonical Architecture for Multivariate Time Series](https://mlsanity.com/paper/2603.08753/) (2026)
- [Topological Data Analysis for High-Dimensional Dynamic Process Monitoring](https://mlsanity.com/paper/2606.20443/) (2026)

## Abstract (arXiv)

Modeling non-Euclidean data is drawing extensive attention along with the unprecedented successes of deep neural networks in diverse fields. Particularly, a symmetric positive definite matrix is being actively studied in computer vision, signal processing, and medical image analysis, due to its ability to learn beneficial statistical representations. However, owing to its rigid constraints, it remains challenging to optimization problems and inefficient computational costs, especially, when incorporating it with a deep learning framework. In this paper, we propose a framework to exploit a diffeomorphism mapping between Riemannian manifolds and a Cholesky space, by which it becomes feasible not only to efficiently solve optimization problems but also to greatly reduce computation costs. Further, for dynamic modeling of time-series data, we devise a continuous manifold learning method by systematically integrating a manifold ordinary differential equation and a gated recurrent neural network. It is worth noting that due to the nice parameterization of matrices in a Cholesky space, training our proposed network equipped with Riemannian geometric metrics is straightforward. We demonstrate through experiments over regular and irregular time-series datasets that our proposed model can be efficiently and reliably trained and outperforms existing manifold methods and state-of-the-art methods in various time-series tasks.