mlsanityData to 7 Oct 2026

Tabular Transformers for Modeling Multivariate Time Series

arXiv 2011.01843 ↗First posted 3 Nov 2020cs.LG cs.AIcited by 77

Not on the map. Why?

Abstract (arXiv)

Tabular datasets are ubiquitous in data science applications. Given their importance, it seems natural to apply state-of-the-art deep learning algorithms in order to fully unlock their potential. Here we propose neural network models that represent tabular time series that can optionally leverage their hierarchical structure. This results in two architectures for tabular time series: one for learning representations that is analogous to BERT and can be pre-trained end-to-end and used in downstream tasks, and one that is akin to GPT and can be used for generation of realistic synthetic tabular sequences. We demonstrate our models on two datasets: a synthetic credit card transaction dataset, where the learned representations are used for fraud detection and synthetic data generation, and on a real pollution dataset, where the learned encodings are used to predict atmospheric pollutant concentrations. Code and data are available at https://github.com/IBM/TabFormer.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 10; 2023: 17; 2024: 17; 2025: 18; 2026: 12.

77 papers.

Sort:

By area:

  1. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  2. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  3. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  4. 2022TransTab: Learning Transferable Tabular Transformers Across Tablescited by 132
  5. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 103
  6. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 93
  7. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 81
  8. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  9. 2022Transformers in Time-series Analysis: A Tutorialcited by 45
  10. 2023TabuLa: Harnessing Language Models for Tabular Data Synthesiscited by 42
  11. 2023TabMT: Generating tabular data with masked transformerscited by 36
  12. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
38 more
  1. 2021TGRNet: A Table Graph Reconstruction Network for Table Structure Recognitioncited by 27
  2. 2025Large Language Models are Good Relational Learnerscited by 22
  3. 2024BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Modelcited by 21
  4. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
  5. 2024Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolutioncited by 19
  6. 2023FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Datacited by 14
  7. 2023Auditing and Generating Synthetic Data with Controllable Trust Trade-offscited by 13
  8. 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patternscited by 13
  9. 2023One Transformer for All Time Series: Representing and Training with Time-Dependent Heterogeneous Tabular Datacited by 12
  10. 2024HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?cited by 11
  11. 2024Multimodal Banking Dataset: Understanding Client Needs through Event Sequencescited by 9
  12. 2022LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-trainingcited by 8
  13. 2024LaTable: Towards Large Tabular Modelscited by 7
  14. 2022Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systemscited by 6
  15. 2023AllSpark: A Multimodal Spatio-Temporal General Intelligence Model with Ten Modalities via Language as a Reference Frameworkcited by 6
  16. 2025Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generationcited by 6
  17. 2023Visual-Attribute Prompt Learning for Progressive Mild Cognitive Impairment Predictioncited by 5
  18. 2023GeoFormer: Predicting Human Mobility using Generative Pre-trained Transformer (GPT)cited by 5
  19. 2024Improve Fidelity and Utility of Synthetic Credit Card Transaction Time Series from Data-centric Perspectivecited by 5
  20. 2024Learning Transactions Representations for Information Management in Banks: Mastering Local, Global, and External Knowledgecited by 5
  21. 2024Detecting the Future: All-at-Once Event Sequence Forecasting with Horizon Matchingcited by 5
  22. 2025New Money: A Systematic Review of Synthetic Data Generation for Financecited by 5
  23. 2021Attention Augmented Convolutional Transformer for Tabular Time-seriescited by 4
  24. 2022The GatedTabTransformer. An enhanced deep learning architecture for tabular modelingcited by 4
  25. 2024Explore the Limits of Omni-modal Pretraining at Scalecited by 4
  26. 2024TimeAutoDiff: A Unified Framework for Generation, Imputation, Forecasting, and Time-Varying Metadata Conditioning of Heterogeneous Time Series Tabular Datacited by 4
  27. 2025Diffusion Transformers for Tabular Data Time Series Generationcited by 4
  28. 2025PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modelingcited by 4
  29. 2022Generative Modeling of Complex Datacited by 3
  30. 2023TimelyGPT: Extrapolatable Transformer Pre-training for Long-term Time-Series Forecasting in Healthcarecited by 3
  31. 2025GReaTER: Generate Realistic Tabular data after data Enhancement and Reductioncited by 3
  32. 2025HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokenscited by 3
  33. 2022NLP Based Anomaly Detection for Categorical Time Seriescited by 2
  34. 2022Masked Multi-Step Multivariate Time Series Forecasting with Future Informationcited by 2
  35. 2024TabularFM: An Open Framework For Tabular Foundational Modelscited by 2
  36. 2024TIMeSynC: Temporal Intent Modelling with Synchronized Context Encodings for Financial Service Applicationscited by 2
  37. 2024DEREC-SIMPRO: unlock Language Model benefits to advance Synthesis in Data Clean Roomcited by 2
  38. 2025Improving Credit Card Fraud Detection through Transformer-Enhanced GAN Oversamplingcited by 2

50 most cited of 77 All 77 (JSON)