# Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification

arXiv: https://arxiv.org/abs/2306.13856 · HTML: https://mlsanity.com/paper/2306.13856/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 24 Jun 2023.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining](https://mlsanity.com/paper/2208.12262/) (2022)
- [CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels](https://mlsanity.com/paper/2211.13977/) (2022)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models](https://mlsanity.com/paper/2301.06267/) (2023)
- [Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression](https://mlsanity.com/paper/2103.13629/) (2021)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Improving Deep Regression with Ordinal Entropy](https://mlsanity.com/paper/2301.08915/) (2023)
- [Moving Window Regression: A Novel Approach to Ordinal Regression](https://mlsanity.com/paper/2203.13122/) (2022)
- [OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression](https://mlsanity.com/paper/2206.02338/) (2022)
- [BridgeNet: A Continuity-Aware Probabilistic Network for Age Estimation](https://mlsanity.com/paper/1904.03358/) (2019)
- [Deep Regression Forests for Age Estimation](https://mlsanity.com/paper/1712.07195/) (2017)
- [PML: Progressive Margin Loss for Long-tailed Age Classification](https://mlsanity.com/paper/2103.02140/) (2021)
- [Unimodal-Concentrated Loss: Fully Adaptive Label Distribution Learning for Ordinal Regression](https://mlsanity.com/paper/2204.00309/) (2022)
- [An Image is Worth More than a Thousand Favorites: Surfacing the Hidden Beauty of Flickr Pictures](https://mlsanity.com/paper/1505.03358/) (2015)
- [Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation](https://mlsanity.com/paper/2202.05167/) (2022)
- [Pluralistic Aging Diffusion Autoencoder](https://mlsanity.com/paper/2303.11086/) (2023)
- [Global and Local Consistent Wavelet-domain Age Synthesis](https://mlsanity.com/paper/1809.07764/) (2018)

## Built on by (most cited first, 21 of 21)

- [Teach CLIP to Develop a Number Sense for Ordinal Regression](https://mlsanity.com/paper/2408.03574/) (2024)
- [InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser](https://mlsanity.com/paper/2311.15040/) (2023)
- [CILF-CIAE: CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation](https://mlsanity.com/paper/2312.01758/) (2023)
- [VisNumBench: Evaluating Number Sense of Multimodal Large Language Models](https://mlsanity.com/paper/2503.14939/) (2025)
- [CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss](https://mlsanity.com/paper/2504.17813/) (2025)
- [Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation](https://mlsanity.com/paper/2403.02302/) (2024)
- [LG-Gaze: Learning Geometry-aware Continuous Prompts for Language-Guided Gaze Estimation](https://mlsanity.com/paper/2411.08606/) (2024)
- [A Survey on Ordinal Regression: Applications, Advances and Prospects](https://mlsanity.com/paper/2503.00952/) (2025)
- [On the rankability of visual embeddings](https://mlsanity.com/paper/2507.03683/) (2025)
- [Ordinal Label-Distribution Learning with Constrained Asymmetric Priors for Imbalanced Retinal Grading](https://mlsanity.com/paper/2509.26146/) (2025)
- [OrderChain: Towards General Instruct-Tuning for Stimulating the Ordinal Understanding Ability of MLLM](https://mlsanity.com/paper/2504.04801/) (2025)
- [Training-free Stylized Text-to-Image Generation with Fast Inference](https://mlsanity.com/paper/2505.19063/) (2025)
- [Exploring 3D-aware Lifespan Face Aging via Disentangled Shape-Texture Representations](https://mlsanity.com/paper/2312.16881/) (2023)
- [Ranking-aware adapter for text-driven image ordering with CLIP](https://mlsanity.com/paper/2412.06760/) (2024)
- [Ranking-aware Reinforcement Learning for Ordinal Ranking](https://mlsanity.com/paper/2601.20585/) (2026)
- [FluoCLIP: Stain-Aware Focus Quality Assessment in Fluorescence Microscopy](https://mlsanity.com/paper/2602.23791/) (2026)
- [DiffoR: A Unified Continuous Generative Framework for Universal Ordinal Regression](https://mlsanity.com/paper/2606.07599/) (2026)
- [TSPRank: Bridging Pairwise and Listwise Methods with a Bilinear Travelling Salesman Model](https://mlsanity.com/paper/2411.12064/) (2024)
- [STORM: Benchmarking Visual Rating of MLLMs with a Comprehensive Ordinal Regression Dataset](https://mlsanity.com/paper/2506.01738/) (2025)
- [DiverAge: Reliable Pluralistic Face Aging with Cross-Age Identity Relation Guidance](https://mlsanity.com/paper/2606.04881/) (2026)
- [Contrastive Order Learning: A General Framework for Ordinal Regression](https://mlsanity.com/paper/2607.08109/) (2026)

## Abstract (arXiv)

We present a novel language-driven ordering alignment method for ordinal classification. The labels in ordinal classification contain additional ordering relations, making them prone to overfitting when relying solely on training data. Recent developments in pre-trained vision-language models inspire us to leverage the rich ordinal priors in human language by converting the original task into a visionlanguage alignment task. Consequently, we propose L2RCLIP, which fully utilizes the language priors from two perspectives. First, we introduce a complementary prompt tuning technique called RankFormer, designed to enhance the ordering relation of original rank prompts. It employs token-level attention with residual-style prompt blending in the word embedding space. Second, to further incorporate language priors, we revisit the approximate bound optimization of vanilla cross-entropy loss and restructure it within the cross-modal embedding space. Consequently, we propose a cross-modal ordinal pairwise loss to refine the CLIP feature space, where texts and images maintain both semantic alignment and ordering alignment. Extensive experiments on three ordinal classification tasks, including facial age estimation, historical color image (HCI) classification, and aesthetic assessment demonstrate its promising performance. The code is available at https://github.com/raywang335/L2RCLIP.