# Field-wise Learning for Multi-field Categorical Data

arXiv: https://arxiv.org/abs/2012.00202 · HTML: https://mlsanity.com/paper/2012.00202/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 6 of 6)

- [Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach](https://mlsanity.com/paper/2110.04514/) (2021)
- [Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation](https://mlsanity.com/paper/2102.10935/) (2021)
- [ParsNets: A Parsimonious Orthogonal and Low-Rank Linear Networks for Zero-Shot Learning](https://mlsanity.com/paper/2312.09709/) (2023)
- [Exploiting Field Dependencies for Learning on Categorical Data](https://mlsanity.com/paper/2307.09321/) (2023)
- [Latent Evolution Model for Change Point Detection in Time-varying Networks](https://mlsanity.com/paper/2212.08818/) (2022)
- [Exploiting Web Images for Fine-Grained Visual Recognition by Eliminating Noisy Samples and Utilizing Hard Ones](https://mlsanity.com/paper/2101.09412/) (2021)

## Abstract (arXiv)

We propose a new method for learning with multi-field categorical data. Multi-field categorical data are usually collected over many heterogeneous groups. These groups can reflect in the categories under a field. The existing methods try to learn a universal model that fits all data, which is challenging and inevitably results in learning a complex model. In contrast, we propose a field-wise learning method leveraging the natural structure of data to learn simple yet efficient one-to-one field-focused models with appropriate constraints. In doing this, the models can be fitted to each category and thus can better capture the underlying differences in data. We present a model that utilizes linear models with variance and low-rank constraints, to help it generalize better and reduce the number of parameters. The model is also interpretable in a field-wise manner. As the dimensionality of multi-field categorical data can be very high, the models applied to such data are mostly over-parameterized. Our theoretical analysis can potentially explain the effect of over-parametrization on the generalization of our model. It also supports the variance constraints in the learning objective. The experiment results on two large-scale datasets show the superior performance of our model, the trend of the generalization error bound, and the interpretability of learning outcomes. Our code is available at https://github.com/lzb5600/Field-wise-Learning.