# Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques

arXiv: https://arxiv.org/abs/2405.11775 · HTML: https://mlsanity.com/paper/2405.11775/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Do Prompt-Based Models Really Understand the Meaning of their Prompts?](https://mlsanity.com/paper/2109.01247/) (2021)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [Rank consistent ordinal regression for neural networks with application to age estimation](https://mlsanity.com/paper/1901.07884/) (2019)
- [Entailment as Few-Shot Learner](https://mlsanity.com/paper/2104.14690/) (2021)
- [Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks](https://mlsanity.com/paper/2002.10553/) (2020)
- [Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis](https://mlsanity.com/paper/2105.01650/) (2021)
- [Sentiment Analysis Based on Deep Learning: A Comparative Study](https://mlsanity.com/paper/2006.03541/) (2020)
- [Unimodal probability distributions for deep ordinal classification](https://mlsanity.com/paper/1705.05278/) (2017)
- [Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning](https://mlsanity.com/paper/1904.03673/) (2019)
- [Non-asymptotic convergence bounds for Wasserstein approximation using point clouds](https://mlsanity.com/paper/2106.07911/) (2021)
- [On Robustness of Finetuned Transformer-based NLP Models](https://mlsanity.com/paper/2305.14453/) (2023)
- [Yelp Review Rating Prediction: Machine Learning and Deep Learning Models](https://mlsanity.com/paper/2012.06690/) (2020)

## Built on by (most cited first, 2 of 2)

- [Generative or Discriminative? Revisiting Text Classification in the Era of Transformers](https://mlsanity.com/paper/2506.12181/) (2025)
- [The Hidden Cost of Modeling P(X): Vulnerability to Membership Inference Attacks in Generative Text Classifiers](https://mlsanity.com/paper/2510.16122/) (2025)

## Abstract (arXiv)

Ordinal Classification (OC) is a widely encountered challenge in Natural Language Processing (NLP), with applications in various domains such as sentiment analysis, rating prediction, and more. Previous approaches to tackle OC have primarily focused on modifying existing or creating novel loss functions that explicitly account for the ordinal nature of labels. However, with the advent of Pretrained Language Models (PLMs), it became possible to tackle ordinality through the implicit semantics of the labels as well. This paper provides a comprehensive theoretical and empirical examination of both these approaches. Furthermore, we also offer strategic recommendations regarding the most effective approach to adopt based on specific settings.