Scaling Laws, Tabular Data and Actuarial Ratemaking Models
Not on the map. Why?
Abstract (arXiv)
Scaling laws in modern deep learning describe how held-out loss improves as model capacity, training data, and compute increase, often following power-law trends. We investigate whether analogous scaling regularities arise in actuarial ratemaking, where data are tabular, heterogeneous, and noisy, and where classical models such as GLMs remain strong baselines. Using a real-world motor insurance portfolio, we train models from different families across increasing fractions of the training data and multiple random seeds, evaluating out-of-sample Poisson deviance, a likelihood-based loss for Poisson count predictions in which lower values indicate better held-out fit. We find that all model families improve with additional data, but scaling exponents differ substantially: TabM exhibits markedly stronger data scaling than purely supervised tabular Transformers and standard MLP baselines. Transformer variants show weak parameter scaling unless augmented with additional inductive biases (TabM-style adaptation or self-supervision). These results provide quantitative guidance on model selection by data regime and suggest that effective scaling on actuarial tabular tasks depends on architecture and loss function objective design, with simple increases in Transformer size providing limited gains.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 60 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
31 of 60 references.
References · 31
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2017Decoupled Weight Decay Regularizationcited by 1,831
- 2019Root Mean Square Layer Normalizationcited by 1,480
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
- 2021Going deeper with Image Transformerscited by 764
- 2021Revisiting Deep Learning Models for Tabular Datacited by 740
- 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
19 more
- 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
- 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
- 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
- 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
- 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
- 2021Transformers Can Do Bayesian Inferencecited by 360
- 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
- 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 223
- 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
- 2018Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networkscited by 132
- 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
- 2022Broken Neural Scaling Lawscited by 109
- 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 92
- 2016Entity Embeddings of Categorical Variablescited by 87
- 2017CatBoost: unbiased boosting with categorical featurescited by 31
- 2024A Hitchhiker's Guide to Scaling Law Estimationcited by 27
- 2024Value Residual Learningcited by 27
- 2021LocalGLMnet: interpretable deep learning for tabular datacited by 15
- 2023Isotonic Recalibration under a Low Signal-to-Noise Ratiocited by 8
Built on by
Not cited yet.