# Learning Deep Time-index Models for Time Series Forecasting

arXiv: https://arxiv.org/abs/2207.06046 · HTML: https://mlsanity.com/paper/2207.06046/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 13 Jul 2022.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Learned Initializations for Optimizing Coordinate-Based Neural Representations](https://mlsanity.com/paper/2012.02189/) (2020)
- [MetaSDF: Meta-learning Signed Distance Functions](https://mlsanity.com/paper/2006.09662/) (2020)
- [ETSformer: Exponential Smoothing Transformers for Time-series Forecasting](https://mlsanity.com/paper/2202.01381/) (2022)
- [N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting](https://mlsanity.com/paper/2201.12886/) (2022)
- [Frequency Bias in Neural Networks for Input of Non-Uniform Density](https://mlsanity.com/paper/2003.04560/) (2020)
- [A Structured Dictionary Perspective on Implicit Neural Representations](https://mlsanity.com/paper/2112.01917/) (2021)
- [Meta-Learning without Memorization](https://mlsanity.com/paper/1912.03820/) (2019)
- [Generative Models as Distributions of Functions](https://mlsanity.com/paper/2102.04776/) (2021)
- [Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory](https://mlsanity.com/paper/1711.01244/) (2017)
- [Meta-Learning Requires Meta-Augmentation](https://mlsanity.com/paper/2007.05549/) (2020)
- [Time series forecasting with Gaussian Processes needs priors](https://mlsanity.com/paper/2009.08102/) (2020)
- [Time-Series Anomaly Detection with Implicit Neural Representation](https://mlsanity.com/paper/2201.11950/) (2022)
- [Neural Decomposition of Time-Series Data for Effective Generalization](https://mlsanity.com/paper/1705.09137/) (2017)

## Built on by (most cited first, 18 of 18)

- [Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretability](https://mlsanity.com/paper/2406.02496/) (2024)
- [A Survey on Deep Learning based Time Series Analysis with Frequency Transformation](https://mlsanity.com/paper/2302.02173/) (2023)
- [Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain](https://mlsanity.com/paper/2310.05063/) (2023)
- [Are Self-Attentions Effective for Time Series Forecasting?](https://mlsanity.com/paper/2405.16877/) (2024)
- [Quantum Implicit Neural Representations](https://mlsanity.com/paper/2406.03873/) (2024)
- [Spatiotemporal Implicit Neural Representation as a Generalized Traffic Data Learner](https://mlsanity.com/paper/2405.03185/) (2024)
- [A Joint Time-frequency Domain Transformer for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2305.14649/) (2023)
- [TS-ICL: A Flexible Time-Indexed Foundation Model for Time Series via In-Context Learning](https://mlsanity.com/paper/2606.05878/) (2026)
- [Partial-Multivariate Model for Forecasting](https://mlsanity.com/paper/2408.09703/) (2024)
- [DynaConF: Dynamic Forecasting of Non-Stationary Time Series](https://mlsanity.com/paper/2209.08411/) (2022)
- [PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations](https://mlsanity.com/paper/2402.16913/) (2024)
- [Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift](https://mlsanity.com/paper/2407.13194/) (2024)
- [Temporal Variational Implicit Neural Representations](https://mlsanity.com/paper/2506.01544/) (2025)
- [MoTM: Towards a Foundation Model for Time Series Imputation based on Continuous Modeling](https://mlsanity.com/paper/2507.13207/) (2025)
- [Are Time-Indexed Foundation Models the Future of Time Series Imputation?](https://mlsanity.com/paper/2511.05980/) (2025)
- [Variational quantization for state space models](https://mlsanity.com/paper/2404.11117/) (2024)
- [Collaborative Imputation of Urban Time Series through Cross-city Meta-learning](https://mlsanity.com/paper/2501.11306/) (2025)
- [iADCPS: Time Series Anomaly Detection for Evolving Cyber-physical Systems via Incremental Meta-learning](https://mlsanity.com/paper/2504.04374/) (2025)

## Abstract (arXiv)

Deep learning has been actively applied to time series forecasting, leading to a deluge of new methods, belonging to the class of historical-value models. Yet, despite the attractive properties of time-index models, such as being able to model the continuous nature of underlying time series dynamics, little attention has been given to them. Indeed, while naive deep time-index models are far more expressive than the manually predefined function representations of classical time-index models, they are inadequate for forecasting, being unable to generalize to unseen time steps due to the lack of inductive bias. In this paper, we propose DeepTime, a meta-optimization framework to learn deep time-index models which overcome these limitations, yielding an efficient and accurate forecasting model. Extensive experiments on real world datasets in the long sequence time-series forecasting setting demonstrate that our approach achieves competitive results with state-of-the-art methods, and is highly efficient. Code is available at https://github.com/salesforce/DeepTime.