# Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms

arXiv: https://arxiv.org/abs/1805.09717 · HTML: https://mlsanity.com/paper/1805.09717/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2018.

## Built on by (most cited first, 27 of 27)

- [DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning](https://mlsanity.com/paper/2106.03760/) (2021)
- [CaDA: Cross-Problem Routing Solver with Constraint-Aware Dual-Attention](https://mlsanity.com/paper/2412.00346/) (2024)
- [From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Module](https://mlsanity.com/paper/2305.16174/) (2023)
- [AdaSplash: Adaptive Sparse Flash Attention](https://mlsanity.com/paper/2502.12082/) (2025)
- [Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences](https://mlsanity.com/paper/2305.00135/) (2023)
- [MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts](https://mlsanity.com/paper/2405.18979/) (2024)
- [Contrasting quadratic assignments for set-based representation learning](https://mlsanity.com/paper/2205.15814/) (2022)
- [MonarchRT: Efficient Attention for Real-Time Video Generation](https://mlsanity.com/paper/2602.12271/) (2026)
- [MonarchAttention: Zero-Shot Conversion to Fast, Hardware-Aware Structured Attention](https://mlsanity.com/paper/2505.18698/) (2025)
- [DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention](https://mlsanity.com/paper/2605.18753/) (2026)
- [Learning with Fitzpatrick Losses](https://mlsanity.com/paper/2405.14574/) (2024)
- [AdaSplash-2: Faster Differentiable Sparse Attention](https://mlsanity.com/paper/2604.15180/) (2026)
- [Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognition](https://mlsanity.com/paper/2209.15176/) (2022)
- [Robust Classification with Noisy Labels Based on Posterior Maximization](https://mlsanity.com/paper/2504.06805/) (2025)
- [Entropy-Generated Attention Beyond Softmax and Entmax: Kaniadakis and Reciprocal-Symmetric Abe Operators](https://mlsanity.com/paper/2602.08216/) (2026)
- [Memory Wrap: a Data-Efficient and Interpretable Extension to Image Classification Models](https://mlsanity.com/paper/2106.01440/) (2021)
- [Fenchel-Young Variational Learning](https://mlsanity.com/paper/2502.10295/) (2025)
- [Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems](https://mlsanity.com/paper/2509.15448/) (2025)
- [Differentiable Knapsack and Top-k Operators via Dynamic Programming](https://mlsanity.com/paper/2601.21775/) (2026)
- [On Inductive Biases for Machine Learning in Data Constrained Settings](https://mlsanity.com/paper/2302.10692/) (2023)
- [A Fenchel-Young Loss Approach to Data-Driven Inverse Optimization](https://mlsanity.com/paper/2502.16120/) (2025)
- [Weighted Graph Structure Learning with Attention Denoising for Node Classification](https://mlsanity.com/paper/2503.12157/) (2025)
- [Alpha Divergence Losses for Biometric Verification](https://mlsanity.com/paper/2511.13621/) (2025)
- [Weighted Graph Clustering via Scale Contraction and Graph Structure Learning](https://mlsanity.com/paper/2601.17307/) (2026)
- [EntmaxKV: Support-Aware Decoding for Entmax Attention](https://mlsanity.com/paper/2605.21649/) (2026)
- [Sparsity-Inducing Divergence Losses for Biometric Verification](https://mlsanity.com/paper/2606.31664/) (2026)
- [Sparse Attention for Dense Open-Vocabulary Prediction in CLIP](https://mlsanity.com/paper/2607.07135/) (2026)

## Abstract (arXiv)

This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create useful new ones easily. Fenchel-Young losses constructed from a generalized entropy, including the Shannon and Tsallis entropies, induce predictive probability distributions. We formulate conditions for a generalized entropy to yield losses with a separation margin, and probability distributions with sparse support. Finally, we derive efficient algorithms, making Fenchel-Young losses appealing both in theory and practice.