# The Maximum von Neumann Entropy Principle: Theory and Applications in Machine Learning

arXiv: https://arxiv.org/abs/2602.02117 · HTML: https://mlsanity.com/paper/2602.02117/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Information Theory with Kernel Methods](https://mlsanity.com/paper/2202.08545/) (2022)
- [Unicom: Universal and Compact Representation Learning for Image Retrieval](https://mlsanity.com/paper/2304.05884/) (2023)
- [A Minimax Approach to Supervised Learning](https://mlsanity.com/paper/1606.02206/) (2016)
- [Towards a Scalable Reference-Free Evaluation of Generative Models](https://mlsanity.com/paper/2407.02961/) (2024)
- [Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learning](https://mlsanity.com/paper/2310.12952/) (2023)
- [An Interpretable Evaluation of Entropy-based Novelty of Generative Models](https://mlsanity.com/paper/2402.17287/) (2024)
- [Unveiling Differences in Generative Models: A Scalable Differential Clustering Approach](https://mlsanity.com/paper/2405.02700/) (2024)
- [Scendi Score: Prompt-Aware Diversity Evaluation via Schur Complement of CLIP Embeddings](https://mlsanity.com/paper/2412.18645/) (2024)
- [Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Models](https://mlsanity.com/paper/2506.02557/) (2025)
- [Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithms](https://mlsanity.com/paper/2412.17622/) (2024)
- [SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score](https://mlsanity.com/paper/2506.10173/) (2025)
- [A Multi-Armed Bandit Approach to Online Selection and Evaluation of Generative Models](https://mlsanity.com/paper/2406.07451/) (2024)
- [PAK-UCB Contextual Bandit: An Online Learning Approach to Prompt-Aware Selection of Generative Models and LLMs](https://mlsanity.com/paper/2410.13287/) (2024)
- [Towards an Explainable Comparison and Alignment of Feature Embeddings](https://mlsanity.com/paper/2506.06231/) (2025)
- [Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications](https://mlsanity.com/paper/1805.11769/) (2018)
- [Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs](https://mlsanity.com/paper/2411.02817/) (2024)
- [When Kernels Multiply, Clusters Unify: Fusing Embeddings with the Kronecker Product](https://mlsanity.com/paper/2506.08645/) (2025)
- [Discrete R'enyi Classifiers](https://mlsanity.com/paper/1511.01764/) (2015)

## Built on by (most cited first, 3 of 3)

- [KODA: Contrastive Representation Comparison and Alignment for Vision-Language Foundation Models](https://mlsanity.com/paper/2606.04180/) (2026)
- [Mixture-Greedy for Online Generative Model Selection: Is UCB Necessary in Diversity-Aware Multi-Armed Bandits?](https://mlsanity.com/paper/2603.21716/) (2026)
- [Exploring More to Solve More: Boosting Diversity in Text Diffusion Models via Entropy-Based Guidance](https://mlsanity.com/paper/2608.00024/) (2026)

## Abstract (arXiv)

Von Neumann entropy (VNE) is a fundamental quantity in quantum information theory and has recently been adopted in machine learning as a spectral measure of diversity for kernel matrices and kernel covariance operators. While maximizing VNE under constraints is well known in quantum settings, a principled analogue of the classical maximum entropy framework, particularly its decision theoretic and game theoretic interpretation, has not been explicitly developed for VNE in data driven contexts. In this paper, we extend the minimax formulation of the maximum entropy principle due to Gr"unwald and Dawid to the setting of von Neumann entropy, providing a game-theoretic justification for VNE maximization over density matrices and trace-normalized positive semidefinite operators. This perspective yields a robust interpretation of maximum VNE solutions under partial information and clarifies their role as least committed inferences in spectral domains. We then illustrate how the resulting Maximum VNE principle applies to modern machine learning problems by considering two representative applications, selecting a kernel representation from multiple normalized embeddings via kernel-based VNE maximization, and completing kernel matrices from partially observed entries. These examples demonstrate how the proposed framework offers a unifying information-theoretic foundation for VNE-based methods in kernel learning.