# TACTiS: Transformer-Attentional Copulas for Time Series

arXiv: https://arxiv.org/abs/2202.03528 · HTML: https://mlsanity.com/paper/2202.03528/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 80 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [Autoregressive Diffusion Models](https://mlsanity.com/paper/2110.02037/) (2021)
- [High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes](https://mlsanity.com/paper/1910.03002/) (2019)
- [Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows](https://mlsanity.com/paper/2002.06103/) (2020)
- [Temporal Pattern Attention for Multivariate Time Series Forecasting](https://mlsanity.com/paper/1809.04206/) (2018)
- [Principles and Algorithms for Forecasting Groups of Time Series: Locality and Globality](https://mlsanity.com/paper/2008.00444/) (2020)
- [A Deep and Tractable Density Estimator](https://mlsanity.com/paper/1310.1757/) (2013)
- [Probabilistic Forecasting with Temporal Convolutional Neural Network](https://mlsanity.com/paper/1906.04397/) (2019)
- [A Review of Deep Learning Methods for Irregularly Sampled Medical Time Series Data](https://mlsanity.com/paper/2010.12493/) (2020)
- [Pay Attention to Evolution: Time Series Forecasting with Deep Graph-Evolution Learning](https://mlsanity.com/paper/2008.12833/) (2020)
- [Copula & Marginal Flows: Disentangling the Marginal from its Joint](https://mlsanity.com/paper/1907.03361/) (2019)

## Built on by (most cited first, 30 of 38)

- [PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting](https://mlsanity.com/paper/2210.08964/) (2022)
- [Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2310.08278/) (2023)
- [ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons](https://mlsanity.com/paper/2310.07446/) (2023)
- [MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Process](https://mlsanity.com/paper/2403.05751/) (2024)
- [Out-of-Distribution Representation Learning for Time Series Classification](https://mlsanity.com/paper/2209.07027/) (2022)
- [Copula Conformal Prediction for Multi-step Time Series Forecasting](https://mlsanity.com/paper/2212.03281/) (2022)
- [TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series](https://mlsanity.com/paper/2310.01327/) (2023)
- [DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2308.02282/) (2023)
- [Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain](https://mlsanity.com/paper/2310.05063/) (2023)
- [TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning](https://mlsanity.com/paper/2506.13705/) (2025)
- [Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Survey](https://mlsanity.com/paper/2209.03580/) (2022)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [From Samples to Scenarios: A New Paradigm for Probabilistic Forecasting](https://mlsanity.com/paper/2509.19975/) (2025)
- [LOB-Based Deep Learning Models for Stock Price Trend Prediction: A Benchmark Study](https://mlsanity.com/paper/2308.01915/) (2023)
- [Retrospective Uncertainties for Deep Models using Vine Copulas](https://mlsanity.com/paper/2302.12606/) (2023)
- [Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts](https://mlsanity.com/paper/2304.09836/) (2023)
- [Multivariate Probabilistic Time Series Forecasting with Correlated Errors](https://mlsanity.com/paper/2402.01000/) (2024)
- [Inference and Sampling for Archimax Copulas](https://mlsanity.com/paper/2205.14025/) (2022)
- [MVG-CRPS: A Robust Loss Function for Multivariate Probabilistic Forecasting](https://mlsanity.com/paper/2410.09133/) (2024)
- [Probabilistic Traffic Forecasting with Dynamic Regression](https://mlsanity.com/paper/2301.06650/) (2023)
- [Winner-takes-all for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2506.05515/) (2025)
- [Efficiently Generating Correlated Sample Paths from Multi-step Time Series Foundation Models](https://mlsanity.com/paper/2510.02224/) (2025)
- [Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting](https://mlsanity.com/paper/2407.19784/) (2024)
- [DynaConF: Dynamic Forecasting of Non-Stationary Time Series](https://mlsanity.com/paper/2209.08411/) (2022)
- [Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesis](https://mlsanity.com/paper/2405.20602/) (2024)
- [Spatially-Enhanced Temporal Fusion Transformer: Interpretable Multi-Output Prediction for Parametric Dynamical Systems with Time-Varying Inputs](https://mlsanity.com/paper/2505.00473/) (2025)
- [Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency](https://mlsanity.com/paper/2510.19980/) (2025)
- [A Scalable Inter-edge Correlation Modeling in CopulaGNN for Link Sign Prediction](https://mlsanity.com/paper/2601.19175/) (2026)
- [Probabilistic Circuits for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2604.27814/) (2026)
- [Forecast Collapse in Time-Series Foundation Models](https://mlsanity.com/paper/2608.14106/) (2026)

## Abstract (arXiv)

The estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance. However, the practical utility of such estimates is limited by how accurately they quantify predictive uncertainty. In this work, we address the problem of estimating the joint predictive distribution of high-dimensional multivariate time series. We propose a versatile method, based on the transformer architecture, that estimates joint distributions using an attention-based decoder that provably learns to mimic the properties of non-parametric copulas. The resulting model has several desirable properties: it can scale to hundreds of time series, supports both forecasting and interpolation, can handle unaligned and non-uniformly sampled data, and can seamlessly adapt to missing data during training. We demonstrate these properties empirically and show that our model produces state-of-the-art predictions on multiple real-world datasets.