# Long-term Forecasting using Higher Order Tensor RNNs

arXiv: https://arxiv.org/abs/1711.00073 · HTML: https://mlsanity.com/paper/1711.00073/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 30 of 34)

- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Granger Causality: A Review and Recent Advances](https://mlsanity.com/paper/2105.02675/) (2021)
- [Traffic Flow Forecasting with Spatial-Temporal Graph Diffusion Network](https://mlsanity.com/paper/2110.04038/) (2021)
- [Network of Tensor Time Series](https://mlsanity.com/paper/2102.07736/) (2021)
- [Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators](https://mlsanity.com/paper/2106.02205/) (2021)
- [QTN-VQC: An End-to-End Learning framework for Quantum Neural Networks](https://mlsanity.com/paper/2110.03861/) (2021)
- [FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecasting](https://mlsanity.com/paper/2212.01209/) (2022)
- [Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing](https://mlsanity.com/paper/2203.06031/) (2022)
- [MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data](https://mlsanity.com/paper/2110.14354/) (2021)
- [TCCT: Tightly-Coupled Convolutional Transformer on Time Series Forecasting](https://mlsanity.com/paper/2108.12784/) (2021)
- [Guaranteed Nonconvex Factorization Approach for Tensor Train Recovery](https://mlsanity.com/paper/2401.02592/) (2024)
- [Higher Order Recurrent Space-Time Transformer for Video Action Prediction](https://mlsanity.com/paper/2104.08665/) (2021)
- [Tensor-Train Recurrent Neural Networks for Interpretable Multi-Way Financial Forecasting](https://mlsanity.com/paper/2105.04983/) (2021)
- [LETS Forecast: Learning Embedology for Time Series Forecasting](https://mlsanity.com/paper/2506.06454/) (2025)
- [Video Anomaly Detection via Prediction Network with Enhanced Spatio-Temporal Memory Exchange](https://mlsanity.com/paper/2206.12914/) (2022)
- [A Scalable Factorization Approach for High-Order Structured Tensor Recovery](https://mlsanity.com/paper/2506.16032/) (2025)
- [Towards Better Long-range Time Series Forecasting using Generative Forecasting](https://mlsanity.com/paper/2212.06142/) (2022)
- [WEITS: A Wavelet-enhanced residual framework for interpretable time series forecasting](https://mlsanity.com/paper/2405.10877/) (2024)
- [NVIDIA-UNIBZ Submission for EPIC-KITCHENS-100 Action Anticipation Challenge 2022](https://mlsanity.com/paper/2206.10869/) (2022)
- [Granger Causality using Neural Networks](https://mlsanity.com/paper/2208.03703/) (2022)
- [Hierarchical Joint Graph Learning and Multivariate Time Series Forecasting](https://mlsanity.com/paper/2311.12630/) (2023)
- [Boosting Defect Detection in Manufacturing using Tensor Convolutional Neural Networks](https://mlsanity.com/paper/2401.01373/) (2023)
- [Leveraging 2D Information for Long-term Time Series Forecasting with Vanilla Transformers](https://mlsanity.com/paper/2405.13810/) (2024)
- [Error Analysis of Tensor-Train Cross Approximation](https://mlsanity.com/paper/2207.04327/) (2022)
- [Temporal Saliency Detection Towards Explainable Transformer-based Timeseries Forecasting](https://mlsanity.com/paper/2212.07771/) (2022)
- [Inductive Attention for Video Action Anticipation](https://mlsanity.com/paper/2212.08830/) (2022)
- [conv_einsum: A Framework for Representation and Fast Evaluation of Multilinear Operations in Convolutional Tensorial Neural Networks](https://mlsanity.com/paper/2401.03384/) (2024)
- [Towards Better Long-range Time Series Forecasting using Generative Adversarial Networks](https://mlsanity.com/paper/2110.08770/) (2021)

## Abstract (arXiv)

We present Higher-Order Tensor RNN (HOT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics. Long-term forecasting in such systems is highly challenging, since there exist long-term temporal dependencies, higher-order correlations and sensitivity to error propagation. Our proposed recurrent architecture addresses these issues by learning the nonlinear dynamics directly using higher-order moments and higher-order state transition functions. Furthermore, we decompose the higher-order structure using the tensor-train decomposition to reduce the number of parameters while preserving the model performance. We theoretically establish the approximation guarantees and the variance bound for HOT-RNN for general sequence inputs. We also demonstrate 5% ~ 12% improvements for long-term prediction over general RNN and LSTM architectures on a range of simulated environments with nonlinear dynamics, as well on real-world time series data.