# BELIEF in Dependence: Leveraging Atomic Linearity in Data Bits for Rethinking Generalized Linear Models

arXiv: https://arxiv.org/abs/2210.10852 · HTML: https://mlsanity.com/paper/2210.10852/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 62 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [A modern maximum-likelihood theory for high-dimensional logistic regression](https://mlsanity.com/paper/1803.06964/) (2018)
- [Distribution-free binary classification: prediction sets, confidence intervals and calibration](https://mlsanity.com/paper/2006.10564/) (2020)
- [Assumption-lean inference for generalised linear model parameters](https://mlsanity.com/paper/2006.08402/) (2020)
- [BET on Independence](https://mlsanity.com/paper/1610.05246/) (2016)
- [Innovated interaction screening for high-dimensional nonlinear classification](https://mlsanity.com/paper/1501.01029/) (2015)
- [The xyz algorithm for fast interaction search in high-dimensional data](https://mlsanity.com/paper/1610.05108/) (2016)
- [BEAUTY Powered BEAST](https://mlsanity.com/paper/2103.00674/) (2021)
- [A Fused Latent and Graphical Model for Multivariate Binary Data](https://mlsanity.com/paper/1606.08925/) (2016)
- [Pairwise Nonlinear Dependence Analysis of Genomic Data](https://mlsanity.com/paper/2202.09880/) (2022)

## Built on by (most cited first, 1 of 1)

- [Binary Expansion Group Intersection Network](https://mlsanity.com/paper/2603.24763/) (2026)

## Abstract (arXiv)

Two linearly uncorrelated binary variables must be also independent because non-linear dependence cannot manifest with only two possible states. This inherent linearity is the atom of dependency constituting any complex form of relationship. Inspired by this observation, we develop a framework called binary expansion linear effect (BELIEF) for understanding arbitrary relationships with a binary outcome. Models from the BELIEF framework are easily interpretable because they describe the association of binary variables in the language of linear models, yielding convenient theoretical insight and striking Gaussian parallels. With BELIEF, one may study generalized linear models (GLM) through transparent linear models, providing insight into how the choice of link affects modeling. For example, setting a GLM interaction coefficient to zero does not necessarily lead to the kind of no-interaction model assumption as understood under their linear model counterparts. Furthermore, for a binary response, maximum likelihood estimation for GLMs paradoxically fails under complete separation, when the data are most discriminative, whereas BELIEF estimation automatically reveals the perfect predictor in the data that is responsible for complete separation. We explore these phenomena and provide related theoretical results. We also provide preliminary empirical demonstration of some theoretical results.