# MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attention

arXiv: https://arxiv.org/abs/2009.14799 · HTML: https://mlsanity.com/paper/2009.14799/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2020.

## Built on by (most cited first, 29 of 29)

- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [GIVT: Generative Infinite-Vocabulary Transformers](https://mlsanity.com/paper/2312.02116/) (2023)
- [Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting](https://mlsanity.com/paper/2111.06581/) (2021)
- [Deep Inventory Management](https://mlsanity.com/paper/2210.03137/) (2022)
- [Multivariate Time Series Forecasting with Latent Graph Inference](https://mlsanity.com/paper/2203.03423/) (2022)
- [Multivariate Quantile Function Forecaster](https://mlsanity.com/paper/2202.11316/) (2022)
- [Controlling Neural Networks with Rule Representations](https://mlsanity.com/paper/2106.07804/) (2021)
- [Probabilistic Hierarchical Forecasting with Deep Poisson Mixtures](https://mlsanity.com/paper/2110.13179/) (2021)
- [Deep Learning based Forecasting: a case study from the online fashion industry](https://mlsanity.com/paper/2305.14406/) (2023)
- [MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval Augmentation](https://mlsanity.com/paper/2207.10517/) (2022)
- [End-to-End Probabilistic Framework for Learning with Hard Constraints](https://mlsanity.com/paper/2506.07003/) (2025)
- [Quantformer: from attention to profit with a quantitative transformer trading strategy](https://mlsanity.com/paper/2404.00424/) (2024)
- [Neural Coordination and Capacity Control for Inventory Management](https://mlsanity.com/paper/2410.02817/) (2024)
- [Using Pre-trained LLMs for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2501.06386/) (2025)
- [Learning an Inventory Control Policy with General Inventory Arrival Dynamics](https://mlsanity.com/paper/2310.17168/) (2023)
- [spadesuit SPADE spadesuit Split Peak Attention DEcomposition](https://mlsanity.com/paper/2411.05852/) (2024)
- [LLMForecaster: Improving Seasonal Event Forecasts with Unstructured Textual Data](https://mlsanity.com/paper/2412.02525/) (2024)
- [Dynamic Customer Embeddings for Financial Service Applications](https://mlsanity.com/paper/2106.11880/) (2021)
- [Causal Forecasting for Pricing](https://mlsanity.com/paper/2312.15282/) (2023)
- [t0: A Time-Series Foundation Model for Forecasting with Context](https://mlsanity.com/paper/2609.24559/) (2026)
- [Probability Paths and the Structure of Predictions over Time](https://mlsanity.com/paper/2106.06515/) (2021)
- [GEANN: Scalable Graph Augmentations for Multi-Horizon Time Series Forecasting](https://mlsanity.com/paper/2307.03595/) (2023)
- [F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data](https://mlsanity.com/paper/2406.16221/) (2024)
- [Informer in Algorithmic Investment Strategies on High Frequency Bitcoin Data](https://mlsanity.com/paper/2503.18096/) (2025)
- [Forking-Sequences: Statistically and Computationally Efficient Multi-Horizon Forecasting with Reduced Volatility](https://mlsanity.com/paper/2510.04487/) (2025)
- [Operator Learning for Smoothing and Forecasting](https://mlsanity.com/paper/2603.20359/) (2026)
- [A Novel Deep Learning Model for Hotel Demand and Revenue Prediction amid COVID-19](https://mlsanity.com/paper/2203.04383/) (2022)
- [Outbound Modeling for Inventory Management](https://mlsanity.com/paper/2507.10890/) (2025)
- [SPADE-S: A Sparsity-Robust Foundational Forecaster](https://mlsanity.com/paper/2507.21155/) (2025)

## Abstract (arXiv)

Recent advances in neural forecasting have produced major improvements in accuracy for probabilistic demand prediction. In this work, we propose novel improvements to the current state of the art by incorporating changes inspired by recent advances in Transformer architectures for Natural Language Processing. We develop a novel decoder-encoder attention for context-alignment, improving forecasting accuracy by allowing the network to study its own history based on the context for which it is producing a forecast. We also present a novel positional encoding that allows the neural network to learn context-dependent seasonality functions as well as arbitrary holiday distances. Finally we show that the current state of the art MQ-Forecaster (Wen et al., 2017) models display excess variability by failing to leverage previous errors in the forecast to improve accuracy. We propose a novel decoder-self attention scheme for forecasting that produces significant improvements in the excess variation of the forecast.