# Hidden Markov Chains, Entropic Forward-Backward, and Part-Of-Speech Tagging

arXiv: https://arxiv.org/abs/2005.10629 · HTML: https://mlsanity.com/paper/2005.10629/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2020.

## Built on by (most cited first, 6 of 6)

- [Introducing the Hidden Neural Markov Chain framework](https://mlsanity.com/paper/2102.11038/) (2021)
- [On equivalence between linear-chain conditional random fields and hidden Markov chains](https://mlsanity.com/paper/2111.07376/) (2021)
- [Deriving discriminative classifiers from generative models](https://mlsanity.com/paper/2201.00844/) (2022)
- [Linear chain conditional random fields, hidden Markov models, and related classifiers](https://mlsanity.com/paper/2301.01293/) (2023)
- [Pairwise Markov Chains for Volatility Forecasting](https://mlsanity.com/paper/2411.11838/) (2024)
- [Real-time Pollutant Identification through Optical PM Micro-Sensor](https://mlsanity.com/paper/2503.10724/) (2025)

## Abstract (arXiv)

The ability to take into account the characteristics - also called features - of observations is essential in Natural Language Processing (NLP) problems. Hidden Markov Chain (HMC) model associated with classic Forward-Backward probabilities cannot handle arbitrary features like prefixes or suffixes of any size, except with an independence condition. For twenty years, this default has encouraged the development of other sequential models, starting with the Maximum Entropy Markov Model (MEMM), which elegantly integrates arbitrary features. More generally, it led to neglect HMC for NLP. In this paper, we show that the problem is not due to HMC itself, but to the way its restoration algorithms are computed. We present a new way of computing HMC based restorations using original Entropic Forward and Entropic Backward (EFB) probabilities. Our method allows taking into account features in the HMC framework in the same way as in the MEMM framework. We illustrate the efficiency of HMC using EFB in Part-Of-Speech Tagging, showing its superiority over MEMM based restoration. We also specify, as a perspective, how HMCs with EFB might appear as an alternative to Recurrent Neural Networks to treat sequential data with a deep architecture.