# Deriving discriminative classifiers from generative models

arXiv: https://arxiv.org/abs/2201.00844 · HTML: https://mlsanity.com/paper/2201.00844/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 55 references that are in the map.
- First posted on arXiv 3 Jan 2022.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models](https://mlsanity.com/paper/1810.04261/) (2018)
- [Hidden Markov Chains, Entropic Forward-Backward, and Part-Of-Speech Tagging](https://mlsanity.com/paper/2005.10629/) (2020)
- [Introducing the Hidden Neural Markov Chain framework](https://mlsanity.com/paper/2102.11038/) (2021)
- [Highly Fast Text Segmentation With Pairwise Markov Chains](https://mlsanity.com/paper/2102.11037/) (2021)

## Built on by (most cited first, 1 of 1)

- [Pairwise Markov Chains for Volatility Forecasting](https://mlsanity.com/paper/2411.11838/) (2024)

## Abstract (arXiv)

We deal with Bayesian generative and discriminative classifiers. Given a model distribution p(x, y), with the observation y and the target x, one computes generative classifiers by firstly considering p(x, y) and then using the Bayes rule to calculate p(x | y). A discriminative model is directly given by p(x | y), which is used to compute discriminative classifiers. However, recent works showed that the Bayesian Maximum Posterior classifier defined from the Naive Bayes (NB) or Hidden Markov Chain (HMC), both generative models, can also match the discriminative classifier definition. Thus, there are situations in which dividing classifiers into "generative" and "discriminative" is somewhat misleading. Indeed, such a distinction is rather related to the way of computing classifiers, not to the classifiers themselves. We present a general theoretical result specifying how a generative classifier induced from a generative model can also be computed in a discriminative way from the same model. Examples of NB and HMC are found again as particular cases, and we apply the general result to two original extensions of NB, and two extensions of HMC, one of which being original. Finally, we shortly illustrate the interest of the new discriminative way of computing classifiers in the Natural Language Processing (NLP) framework.