mlsanityData to 14 Oct 2026

One Transformer for All Time Series: Representing and Training with Time-Dependent Heterogeneous Tabular Data

arXiv 2302.06375 ↗First posted 13 Feb 2023cs.LG cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

There is a recent growing interest in applying Deep Learning techniques to tabular data, in order to replicate the success of other Artificial Intelligence areas in this structured domain. Specifically interesting is the case in which tabular data have a time dependence, such as, for instance financial transactions. However, the heterogeneity of the tabular values, in which categorical elements are mixed with numerical items, makes this adaptation difficult. In this paper we propose a Transformer architecture to represent heterogeneous time-dependent tabular data, in which numerical features are represented using a set of frequency functions and the whole network is uniformly trained with a unique loss function.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

28 of 45 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  7. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  11. 2021ViViT: A Video Vision Transformercited by 1,620
  12. 2018On the Spectral Bias of Neural Networkscited by 1,341
16 more
  1. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  2. 2022Transformers in Time Series: A Surveycited by 497
  3. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  4. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  5. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  6. 2022Memorizing Transformerscited by 226
  7. 2022Can Foundation Models Wrangle Your Data?cited by 175
  8. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 113
  9. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 94
  10. 2020Tabular Transformers for Modeling Multivariate Time Seriescited by 77
  11. 2021Adversarial Attacks on Deep Models for Financial Transaction Recordscited by 24
  12. 2022Hopular: Modern Hopfield Networks for Tabular Datacited by 18
  13. 2020Google Dataset Search by the Numberscited by 17
  14. 2022OptEmbed: Learning Optimal Embedding Table for Click-through Rate Predictioncited by 14
  15. 2022LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-trainingcited by 8
  16. 2021Attention Augmented Convolutional Transformer for Tabular Time-seriescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 4; 2026: 2.

13 papers.

Sort:

By area:

  1. 2024Universal Time-Series Representation Learning: A Surveycited by 30
  2. 2024Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolutioncited by 19
  3. 2023FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Datacited by 14
  4. 2025Diffusion Transformers for Tabular Data Time Series Generationcited by 5
  5. 2025FinTRec: Transformer Based Unified Contextual Ads Targeting and Personalization for Financial Applicationscited by 4
  6. 2024TIMeSynC: Temporal Intent Modelling with Synchronized Context Encodings for Financial Service Applicationscited by 2
  7. 2025LATTE: Learning Aligned Transactions and Textual Embeddings for Bank Clientscited by 2
  8. 2024Fine-grained Attention in Hierarchical Transformers for Tabular Time-seriescited by 1
  9. 2024A Simple Baseline for Predicting Events with Auto-Regressive Tabular Transformerscited by 1
  10. 2023Comparative Analysis of Transformers for Modeling Tabular Data: A Casestudy using Industry Scale Datasetcited by 0
  11. 2025Basis Transformers for Multi-Task Tabular Regressioncited by 0
  12. 2026SOHET: Sequence Of Heterogeneous Events Transformer with Self-Supervised Pre-Trainingcited by 0
1 more
  1. 2026Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspectivecited by 0