# Improving usual Naive Bayes classifier performances with Neural Naive Bayes based models

arXiv: https://arxiv.org/abs/2111.07307 · HTML: https://mlsanity.com/paper/2111.07307/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 14 Nov 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [Introducing the Hidden Neural Markov Chain framework](https://mlsanity.com/paper/2102.11038/) (2021)
- [Highly Fast Text Segmentation With Pairwise Markov Chains](https://mlsanity.com/paper/2102.11037/) (2021)

## Abstract (arXiv)

Naive Bayes is a popular probabilistic model appreciated for its simplicity and interpretability. However, the usual form of the related classifier suffers from two major problems. First, as caring about the observations' law, it cannot consider complex features. Moreover, it considers the conditional independence of the observations given the hidden variable. This paper introduces the original Neural Naive Bayes, modeling the parameters of the classifier induced from the Naive Bayes with neural network functions. This allows to correct the first problem. We also introduce new Neural Pooled Markov Chain models, alleviating the independence condition. We empirically study the benefits of these models for Sentiment Analysis, dividing the error rate of the usual classifier by 4.5 on the IMDB dataset with the FastText embedding.