# LIT-LVM: Structured Regularization for Interaction Terms in Linear Predictors using Latent Variable Models

arXiv: https://arxiv.org/abs/2506.15492 · HTML: https://mlsanity.com/paper/2506.15492/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 18 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Higher-Order Factorization Machines](https://mlsanity.com/paper/1607.07195/) (2016)
- [Scalable Interpretability via Polynomials](https://mlsanity.com/paper/2205.14108/) (2022)
- [Tensor Factorization via Matrix Factorization](https://mlsanity.com/paper/1501.07320/) (2015)
- [A Mutually Exciting Latent Space Hawkes Process Model for Continuous-time Networks](https://mlsanity.com/paper/2205.09263/) (2022)
- [Characterizing Polarization in Social Networks using the Signed Relational Latent Distance Model](https://mlsanity.com/paper/2301.09507/) (2023)
- [Scalable Higher-Order Tensor Product Spline Models](https://mlsanity.com/paper/2402.01090/) (2024)
- [Factorization Machines with Regularization for Sparse Feature Interactions](https://mlsanity.com/paper/2010.09225/) (2020)
- [Predicting Kidney Transplant Survival using Multiple Feature Representations for HLAs](https://mlsanity.com/paper/2103.03305/) (2021)
- [Factorized Structured Regression for Large-Scale Varying Coefficient Models](https://mlsanity.com/paper/2205.13080/) (2022)
- [Penalized Interaction Estimation for Ultrahigh Dimensional Quadratic Regression](https://mlsanity.com/paper/1901.07147/) (2019)
- [A Latent Space Model for HLA Compatibility Networks in Kidney Transplantation](https://mlsanity.com/paper/2211.02234/) (2022)

## Abstract (arXiv)

Some of the simplest, yet most frequently used predictors in statistics and machine learning use weighted linear combinations of features. Such linear predictors can model non-linear relationships between features by adding interaction terms corresponding to the products of all pairs of features. We consider the problem of accurately estimating coefficients for interaction terms in linear predictors. We hypothesize that the coefficients for different interaction terms have an approximate low-dimensional structure and represent each feature by a latent vector in a low-dimensional space. This low-dimensional representation can be viewed as a structured regularization approach that further mitigates overfitting in high-dimensional settings beyond standard regularizers such as the lasso and elastic net. We demonstrate that our approach, called LIT-LVM, achieves superior prediction accuracy compared to the elastic net, hierarchical lasso, and factorization machines on a wide variety of simulated and real data, particularly when the number of interaction terms is high compared to the number of samples. LIT-LVM also provides low-dimensional latent representations for features that are useful for visualizing and analyzing their relationships.