# Efficient Feature Interactions with Transformers: Improving User Spending Propensity Predictions in Gaming

arXiv: https://arxiv.org/abs/2409.17077 · HTML: https://mlsanity.com/paper/2409.17077/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 8 of 8)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training](https://mlsanity.com/paper/2106.01342/) (2021)
- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)
- [ExcelFormer: A neural network surpassing GBDTs on tabular data](https://mlsanity.com/paper/2301.02819/) (2023)

## Abstract (arXiv)

Dream11 is a fantasy sports platform that allows users to create their own virtual teams for real-life sports events. We host multiple sports and matches for our 200M+ user base. In this RMG (real money gaming) setting, users pay an entry amount to participate in various contest products that we provide to users. In our current work, we discuss the problem of predicting the user's propensity to spend in a gaming round, so it can be utilized for various downstream applications. e.g. Upselling users by incentivizing them marginally as per their spending propensity, or personalizing the product listing based on the user's propensity to spend. We aim to model the spending propensity of each user based on past transaction data. In this paper, we benchmark tree-based and deep-learning models that show good results on structured data, and we propose a new architecture change that is specifically designed to capture the rich interactions among the input features. We show that our proposed architecture outperforms the existing models on the task of predicting the user's propensity to spend in a gaming round. Our new transformer model surpasses the state-of-the-art FT-Transformer, improving MAE by 2.5% and MSE by 21.8%.