# Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model arXiv: https://arxiv.org/abs/2506.05801 ยท HTML: https://mlsanity.com/paper/2506.05801/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 27 of its 51 references that are in the map. - First posted on arXiv 6 Jun 2025. ## Builds on (most cited first, 27 of 27) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015) - [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020) - [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017) - [A Geometric Analysis of Neural Collapse with Unconstrained Features](https://mlsanity.com/paper/2105.02375/) (2021) - [Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training](https://mlsanity.com/paper/2101.12699/) (2021) - [Neural collapse with unconstrained features](https://mlsanity.com/paper/2011.11619/) (2020) - [Extended Unconstrained Features Model for Exploring Deep Neural Collapse](https://mlsanity.com/paper/2202.08087/) (2022) - [On the Role of Neural Collapse in Transfer Learning](https://mlsanity.com/paper/2112.15121/) (2021) - [Imbalance Trouble: Revisiting Neural-Collapse Geometry](https://mlsanity.com/paper/2208.05512/) (2022) - [Are All Losses Created Equal: A Neural Collapse Perspective](https://mlsanity.com/paper/2210.02192/) (2022) - [Limitations of Neural Collapse for Understanding Generalization in Deep Learning](https://mlsanity.com/paper/2202.08384/) (2022) - [On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers](https://mlsanity.com/paper/2012.05420/) (2020) - [Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model](https://mlsanity.com/paper/2305.13165/) (2023) - [Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Data](https://mlsanity.com/paper/2301.00437/) (2023) - [Neural Collapse for Unconstrained Feature Model under Cross-entropy Loss with Imbalanced Data](https://mlsanity.com/paper/2309.09725/) (2023) - [Linguistic Collapse: Neural Collapse in (Large) Language Models](https://mlsanity.com/paper/2405.17767/) (2024) - [A Law of Data Separation in Deep Learning](https://mlsanity.com/paper/2210.17020/) (2022) - [The Tunnel Effect: Building Data Representations in Deep Neural Networks](https://mlsanity.com/paper/2305.19753/) (2023) - [Reverse Engineering Self-Supervised Learning](https://mlsanity.com/paper/2305.15614/) (2023) - [Understanding and Improving Transfer Learning of Deep Models via Neural Collapse](https://mlsanity.com/paper/2212.12206/) (2022) - [Neural Collapse in the Intermediate Hidden Layers of Classification Neural Networks](https://mlsanity.com/paper/2308.02760/) (2023) - [The Prevalence of Neural Collapse in Neural Multivariate Regression](https://mlsanity.com/paper/2409.04180/) (2024) - [Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Model](https://mlsanity.com/paper/2401.02058/) (2024) - [Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?](https://mlsanity.com/paper/2405.14468/) (2024) - [Neural Collapse in Multi-label Learning with Pick-all-label Loss](https://mlsanity.com/paper/2310.15903/) (2023) ## Built on by (most cited first, 6 of 6) - [DVD: Deterministic Video Depth Estimation with Generative Priors](https://mlsanity.com/paper/2603.12250/) (2026) - [Geometric Analysis of Neural Regression Collapse via Intrinsic Dimension](https://mlsanity.com/paper/2510.01105/) (2025) - [Reducing Class-Wise Performance Disparity via Margin Regularization](https://mlsanity.com/paper/2602.00205/) (2026) - [The Implicit Bias of Depth: From Neural Collapse to Softmax Codes](https://mlsanity.com/paper/2605.23087/) (2026) - [Ordinal Neural Collapse as a Representation Prior for Visual Navigation](https://mlsanity.com/paper/2606.26839/) (2026) - [Understanding Head Geometry and Dynamics in Federated Regression through a Natural Solution Selection Rule: An Unconstrained Feature Model Analysis](https://mlsanity.com/paper/2609.39464/) (2026) ## Abstract (arXiv) A phenomenon known as ''Neural Collapse (NC)'' in deep classification tasks, in which the penultimate-layer features and the final classifiers exhibit an extremely simple geometric structure, has recently attracted considerable attention, with the expectation that it can deepen our understanding of how deep neural networks behave. The Unconstrained Feature Model (UFM) has been proposed to explain NC theoretically, and there emerges a growing body of work that extends NC to tasks other than classification and leverages it for practical applications. In this study, we investigate whether a similar phenomenon arises in deep Ordinal Regression (OR) tasks, via combining the cumulative link model for OR and UFM. We show that a phenomenon we call Ordinal Neural Collapse (ONC) indeed emerges and is characterized by the following three properties: (ONC1) all optimal features in the same class collapse to their within-class mean when regularization is applied; (ONC2) these class means align with the classifier, meaning that they collapse onto a one-dimensional subspace; (ONC3) the optimal latent variables (corresponding to logits or preactivations in classification tasks) are aligned according to the class order, and in particular, in the zero-regularization limit, a highly local and simple geometric relationship emerges between the latent variables and the threshold values. We prove these properties analytically within the UFM framework with fixed threshold values and corroborate them empirically across a variety of datasets. We also discuss how these insights can be leveraged in OR, highlighting the use of fixed thresholds.