# On the inconsistency of separable losses for structured prediction

arXiv: https://arxiv.org/abs/2301.10810 · HTML: https://mlsanity.com/paper/2301.10810/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 25 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Learning with Fenchel-Young Losses](https://mlsanity.com/paper/1901.02324/) (2019)
- [Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency](https://mlsanity.com/paper/1809.01812/) (2018)
- [Efficient Second-Order TreeCRF for Neural Dependency Parsing](https://mlsanity.com/paper/2005.00975/) (2020)
- [Structured Prediction with Projection Oracles](https://mlsanity.com/paper/1910.11369/) (2019)
- [One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilities](https://mlsanity.com/paper/1609.07410/) (2016)
- [A General Theory for Structured Prediction with Smooth Convex Surrogates](https://mlsanity.com/paper/1902.01958/) (2019)
- [Dependency Parsing as Head Selection](https://mlsanity.com/paper/1606.01280/) (2016)
- [Span-based discontinuous constituency parsing: a family of exact chart-based algorithms with time complexities from O(n^6) down to O(n^3)](https://mlsanity.com/paper/2003.13785/) (2020)
- [Consistent Structured Prediction with Max-Min Margin Markov Networks](https://mlsanity.com/paper/2007.01012/) (2020)
- [Partially Supervised Named Entity Recognition via the Expected Entity Ratio Loss](https://mlsanity.com/paper/2108.07216/) (2021)
- [On the Consistency of Max-Margin Losses](https://mlsanity.com/paper/2105.15069/) (2021)

## Built on by (most cited first, 2 of 2)

- [Structural generalization in COGS: Supertagging is (almost) all you need](https://mlsanity.com/paper/2310.14124/) (2023)
- [On graph-based reentrancy-free semantic parsing](https://mlsanity.com/paper/2302.07679/) (2023)

## Abstract (arXiv)

In this paper, we prove that separable negative log-likelihood losses for structured prediction are not necessarily Bayes consistent, or, in other words, minimizing these losses may not result in a model that predicts the most probable structure in the data distribution for a given input. This fact opens the question of whether these losses are well-adapted for structured prediction and, if so, why.