# Toto: Time Series Optimized Transformer for Observability

arXiv: https://arxiv.org/abs/2407.07874 · HTML: https://mlsanity.com/paper/2407.07874/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 23 of 23)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction](https://mlsanity.com/paper/2106.09305/) (2021)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [A Length-Extrapolatable Transformer](https://mlsanity.com/paper/2212.10554/) (2022)
- [SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention](https://mlsanity.com/paper/2402.10198/) (2024)
- [A mixture autoregressive model based on Student's t-distribution](https://mlsanity.com/paper/1805.04010/) (2018)

## Built on by (most cited first, 30 of 48)

- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [Moirai 2.0: When Less Is More for Time Series Forecasting](https://mlsanity.com/paper/2511.11698/) (2025)
- [From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting](https://mlsanity.com/paper/2501.02945/) (2025)
- [Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRs](https://mlsanity.com/paper/2412.16178/) (2024)
- [Toto 2.0: Time Series Forecasting Enters the Scaling Era](https://mlsanity.com/paper/2605.20119/) (2026)
- [Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling](https://mlsanity.com/paper/2603.04791/) (2026)
- [Specialized Foundation Models Struggle to Beat Supervised Baselines](https://mlsanity.com/paper/2411.02796/) (2024)
- [Foundation Models for Time Series: A Survey](https://mlsanity.com/paper/2504.04011/) (2025)
- [CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data](https://mlsanity.com/paper/2508.02879/) (2025)
- [TiRex-2: Generalizing TiRex to Multivariate Data and Streaming](https://mlsanity.com/paper/2607.01204/) (2026)
- [TimeCopilot](https://mlsanity.com/paper/2509.00616/) (2025)
- [FlowState: Sampling-Rate-Equivariant Time-Series Forecasting](https://mlsanity.com/paper/2508.05287/) (2025)
- [Xihe: Scalable Zero-Shot Time Series Learner Via Hierarchical Interleaved Block Attention](https://mlsanity.com/paper/2510.21795/) (2025)
- [MoTime: A Dataset Suite for Multimodal Time Series Forecasting](https://mlsanity.com/paper/2505.15072/) (2025)
- [Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models](https://mlsanity.com/paper/2511.05460/) (2025)
- [Byte Pair Encoding for Efficient Time Series Forecasting](https://mlsanity.com/paper/2505.14411/) (2025)
- [time2time: Causal Intervention in Hidden States to Simulate Rare Events in Time Series Foundation Models](https://mlsanity.com/paper/2509.05801/) (2025)
- [Impermanent: A Live Benchmark for Temporal Generalization in Time Series Forecasting](https://mlsanity.com/paper/2603.08707/) (2026)
- [Reasoning-Aware Training for Time Series Forecasting](https://mlsanity.com/paper/2605.08625/) (2026)
- [EXAONE Finance 1.0: An Attention-free Time Series Foundation Model for Financial Time Series](https://mlsanity.com/paper/2609.04239/) (2026)
- [Goal-Oriented Time-Series Forecasting: Foundation Framework Design](https://mlsanity.com/paper/2504.17493/) (2025)
- [ProbFM: Probabilistic Time Series Foundation Model with Uncertainty Decomposition](https://mlsanity.com/paper/2601.10591/) (2026)
- [OpenMHC: Accelerating the Science of Wearable Foundation Models](https://mlsanity.com/paper/2607.16235/) (2026)
- [t0: A Time-Series Foundation Model for Forecasting with Context](https://mlsanity.com/paper/2609.24559/) (2026)
- [DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters](https://mlsanity.com/paper/2602.06597/) (2026)
- [WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting](https://mlsanity.com/paper/2604.10544/) (2026)
- [RareCP: Regime-Aware Retrieval for Efficient Conformal Prediction](https://mlsanity.com/paper/2605.08857/) (2026)
- [Towards Resilient and Autonomous Networks: A BlueSky Vision on AI-Native 6G](https://mlsanity.com/paper/2605.21395/) (2026)
- [APEX: A Network-Native Time-Series Foundation Model for Forecasting and Anomaly Detection for Wireless Edge Operations](https://mlsanity.com/paper/2606.11553/) (2026)
- [On Subquadratic Architectures: From Applications to Principles](https://mlsanity.com/paper/2606.12364/) (2026)

## Abstract (arXiv)

This technical report describes the Time Series Optimized Transformer for Observability (Toto), a new state of the art foundation model for time series forecasting developed by Datadog. In addition to advancing the state of the art on generalized time series benchmarks in domains such as electricity and weather, this model is the first general-purpose time series forecasting foundation model to be specifically tuned for observability metrics. Toto was trained on a dataset of one trillion time series data points, the largest among all currently published time series foundation models. Alongside publicly available time series datasets, 75% of the data used to train Toto consists of fully anonymous numerical metric data points from the Datadog platform. In our experiments, Toto outperforms existing time series foundation models on observability data. It does this while also excelling at general-purpose forecasting tasks, achieving state-of-the-art zero-shot performance on multiple open benchmark datasets.