mlsanityData to 7 Oct 2026

TripCast: Pre-training of Masked 2D Transformers for Trip Time Series Forecasting

arXiv 2410.18612 ↗First posted 24 Oct 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep learning and pre-trained models have shown great success in time series forecasting. However, in the tourism industry, time series data often exhibit a leading time property, presenting a 2D structure. This introduces unique challenges for forecasting in this sector. In this study, we propose a novel modelling paradigm, TripCast, which treats trip time series as 2D data and learns representations through masking and reconstruction processes. Pre-trained on large-scale real-world data, TripCast notably outperforms other state-of-the-art baselines in in-domain forecasting scenarios and demonstrates strong scalability and transferability in out-domain forecasting scenarios.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

30 of 41 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  11. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  12. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
18 more
  1. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  2. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  3. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  4. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  5. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  6. 2023A decoder-only foundation model for time-series forecastingcited by 650
  7. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  9. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  10. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  11. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  12. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 143
  13. 2024Large Language Models for Time Series: A Surveycited by 113
  14. 2020Meta-learning framework with applications to zero-shot time-series forecastingcited by 70
  15. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  16. 2023Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domaincited by 23
  17. 2021Flight Demand Forecasting with Transformerscited by 3
  18. 2023TSPP: A Unified Benchmarking Tool for Time-series Forecastingcited by 2

Built on by

Not cited yet.