mlsanityData to 7 Oct 2026

The Credibility Transformer

arXiv 2409.16653 ↗First posted 25 Sep 2024cs.LG q-fin.GNcited by 5

Not on the map. Why?

Abstract (arXiv)

Inspired by the large success of Transformers in Large Language Models, these architectures are increasingly applied to tabular data. This is achieved by embedding tabular data into low-dimensional Euclidean spaces resulting in similar structures as time-series data. We introduce a novel credibility mechanism to this Transformer architecture. This credibility mechanism is based on a special token that should be seen as an encoder that consists of a credibility weighted average of prior information and observation based information. We demonstrate that this novel credibility mechanism is very beneficial to stabilize training, and our Credibility Transformer leads to predictive models that are superior to state-of-the-art deep learning models.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

18 of 35 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Layer Normalizationcited by 5,579
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  7. 2020GLU Variants Improve Transformercited by 1,724
  8. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  9. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  10. 2016Language Modeling with Gated Convolutional Networkscited by 688
  11. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 398
  12. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
6 more
  1. 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 87
  2. 2016Entity Embeddings of Categorical Variablescited by 85
  3. 2021NormFormer: Improved Transformer Pretraining with Extra Normalizationcited by 83
  4. 2015Artificial Neural Networks Applied to Taxi Destination Predictioncited by 18
  5. 2021LocalGLMnet: interpretable deep learning for tabular datacited by 14
  6. 2023Enhancing Actuarial Non-Life Pricing Models via Transformerscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Tab-TRM: Tiny Recursive Model for Insurance Pricing on Tabular Datacited by 3
  2. 2025Tree-like Pairwise Interaction Networkscited by 2
  3. 2026From Chain-Ladder to Individual Claims Reservingcited by 0
  4. 2026Is TabPFN the Silver Bullet for Insurance Pricing?cited by 0
  5. 2026Many Optimizers But Only One Training Path: Repeated Resampling for Adaptive Optimizer Selectioncited by 0