mlsanityData to 14 Oct 2026

Scaling Laws, Tabular Data and Actuarial Ratemaking Models

arXiv 2609.03106 ↗First posted 2 Sep 2026cs.LG q-fin.RMcited by 0

Not on the map. Why?

Abstract (arXiv)

Scaling laws in modern deep learning describe how held-out loss improves as model capacity, training data, and compute increase, often following power-law trends. We investigate whether analogous scaling regularities arise in actuarial ratemaking, where data are tabular, heterogeneous, and noisy, and where classical models such as GLMs remain strong baselines. Using a real-world motor insurance portfolio, we train models from different families across increasing fractions of the training data and multiple random seeds, evaluating out-of-sample Poisson deviance, a likelihood-based loss for Poisson count predictions in which lower values indicate better held-out fit. We find that all model families improve with additional data, but scaling exponents differ substantially: TabM exhibits markedly stronger data scaling than purely supervised tabular Transformers and standard MLP baselines. Transformer variants show weak parameter scaling unless augmented with additional inductive biases (TabM-style adaptation or self-supervision). These results provide quantitative guidance on model selection by data regime and suggest that effective scaling on actuarial tabular tasks depends on architecture and loss function objective design, with simple increases in Transformer size providing limited gains.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 60 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

31 of 60 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  7. 2017Decoupled Weight Decay Regularizationcited by 1,831
  8. 2019Root Mean Square Layer Normalizationcited by 1,480
  9. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  10. 2021Going deeper with Image Transformerscited by 764
  11. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  12. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
19 more
  1. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  2. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  3. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  4. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  5. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  6. 2021Transformers Can Do Bayesian Inferencecited by 360
  7. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  8. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
  9. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
  10. 2018Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networkscited by 132
  11. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  12. 2022Broken Neural Scaling Lawscited by 109
  13. 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 92
  14. 2016Entity Embeddings of Categorical Variablescited by 87
  15. 2017CatBoost: unbiased boosting with categorical featurescited by 31
  16. 2024A Hitchhiker's Guide to Scaling Law Estimationcited by 27
  17. 2024Value Residual Learningcited by 27
  18. 2021LocalGLMnet: interpretable deep learning for tabular datacited by 15
  19. 2023Isotonic Recalibration under a Low Signal-to-Noise Ratiocited by 8

Built on by

Not cited yet.