# MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval Augmentation

arXiv: https://arxiv.org/abs/2207.10517 · HTML: https://mlsanity.com/paper/2207.10517/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [A Multi-Horizon Quantile Recurrent Forecaster](https://mlsanity.com/paper/1711.11053/) (2017)
- [MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attention](https://mlsanity.com/paper/2009.14799/) (2020)
- [apricot: Submodular selection for data summarization in Python](https://mlsanity.com/paper/1906.03543/) (2019)
- [Deep Generative Quantile-Copula Models for Probabilistic Forecasting](https://mlsanity.com/paper/1907.10697/) (2019)
- [Probabilistic Hierarchical Forecasting with Deep Poisson Mixtures](https://mlsanity.com/paper/2110.13179/) (2021)
- [Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation](https://mlsanity.com/paper/2105.09235/) (2021)

## Built on by (most cited first, 9 of 9)

- [Retrieval-Augmented Diffusion Models for Time Series Forecasting](https://mlsanity.com/paper/2410.18712/) (2024)
- [Deep Inventory Management](https://mlsanity.com/paper/2210.03137/) (2022)
- [REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning](https://mlsanity.com/paper/2311.00519/) (2023)
- [Deep Learning based Forecasting: a case study from the online fashion industry](https://mlsanity.com/paper/2305.14406/) (2023)
- [RATSF: Empowering Customer Service Volume Management through Retrieval-Augmented Time-Series Forecasting](https://mlsanity.com/paper/2403.04180/) (2024)
- [Channel-wise Retrieval for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2604.05543/) (2026)
- [GEANN: Scalable Graph Augmentations for Multi-Horizon Time Series Forecasting](https://mlsanity.com/paper/2307.03595/) (2023)
- [TAT: Temporal-Aligned Transformer for Multi-Horizon Peak Demand Forecasting](https://mlsanity.com/paper/2507.10349/) (2025)
- [RAxSS: Retrieval-Augmented Sparse Sampling for Explainable Variable-Length Medical Time Series Classification](https://mlsanity.com/paper/2510.02936/) (2025)

## Abstract (arXiv)

Multi-horizon probabilistic time series forecasting has wide applicability to real-world tasks such as demand forecasting. Recent work in neural time-series forecasting mainly focus on the use of Seq2Seq architectures. For example, MQTransformer - an improvement of MQCNN - has shown the state-of-the-art performance in probabilistic demand forecasting. In this paper, we consider incorporating cross-entity information to enhance model performance by adding a cross-entity attention mechanism along with a retrieval mechanism to select which entities to attend over. We demonstrate how our new neural architecture, MQRetNN, leverages the encoded contexts from a pretrained baseline model on the entire population to improve forecasting accuracy. Using MQCNN as the baseline model (due to computational constraints, we do not use MQTransformer), we first show on a small demand forecasting dataset that it is possible to achieve ~3% improvement in test loss by adding a cross-entity attention mechanism where each entity attends to all others in the population. We then evaluate the model with our proposed retrieval methods - as a means of approximating an attention over a large population - on a large-scale demand forecasting application with over 2 million products and observe ~1% performance gain over the MQCNN baseline.