mlsanityData to 14 Oct 2026

On Interpretable Approaches to Cluster, Classify and Represent Multi-Subspace Data via Minimum Lossy Coding Length based on Rate-Distortion Theory

arXiv 2302.10383 ↗First posted 21 Feb 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

To cluster, classify and represent are three fundamental objectives of learning from high-dimensional data with intrinsic structure. To this end, this paper introduces three interpretable approaches, i.e., segmentation (clustering) via the Minimum Lossy Coding Length criterion, classification via the Minimum Incremental Coding Length criterion and representation via the Maximal Coding Rate Reduction criterion. These are derived based on the lossy data coding and compression framework from the principle of rate distortion in information theory. These algorithms are particularly suitable for dealing with finite-sample data (allowed to be sparse or almost degenerate) of mixed Gaussian distributions or subspaces. The theoretical value and attractive features of these methods are summarized by comparison with other learning methods or evaluation criteria. This summary note aims to provide a theoretical guide to researchers (also engineers) interested in understanding 'white-box' machine (deep) learning methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 68 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

19 of 68 references.

References · 19

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  5. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  6. 2016Group Equivariant Convolutional Networkscited by 1,165
  7. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  8. 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 228
  9. 2020Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reductioncited by 142
  10. 2021ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reductioncited by 84
  11. 2020Deep Isometric Learning for Visual Recognitioncited by 39
  12. 2017OL'E: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learningcited by 37
7 more
  1. 2019A Rate-Distortion Framework for Explaining Neural Network Decisionscited by 24
  2. 2021Learning a Self-Expressive Network for Subspace Clusteringcited by 24
  3. 2017How Generative Adversarial Networks and Their Variants Work: An Overviewcited by 17
  4. 2022Incremental Learning of Structured Memory via Closed-Loop Transcriptioncited by 17
  5. 2020Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimationcited by 13
  6. 2015LogDet Rank Minimization with Application to Subspace Clusteringcited by 6
  7. 2022Efficient Maximal Coding Rate Reduction by Variational Formscited by 6

Built on by

Not cited yet.