# Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation

arXiv: https://arxiv.org/abs/2402.18191 · HTML: https://mlsanity.com/paper/2402.18191/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback](https://mlsanity.com/paper/2309.00267/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [LESS: Selecting Influential Data for Targeted Instruction Tuning](https://mlsanity.com/paper/2402.04333/) (2024)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning](https://mlsanity.com/paper/2312.15685/) (2023)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning](https://mlsanity.com/paper/2308.12032/) (2023)
- [How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition](https://mlsanity.com/paper/2310.05492/) (2023)
- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [MoDS: Model-oriented Data Selection for Instruction Tuning](https://mlsanity.com/paper/2311.15653/) (2023)
- [Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation](https://mlsanity.com/paper/2308.01240/) (2023)
- [A Data-Centric Approach for Training Deep Neural Networks with Less Data](https://mlsanity.com/paper/2110.03613/) (2021)
- [Augmenting Large Language Model Translators via Translation Memories](https://mlsanity.com/paper/2305.17367/) (2023)
- [CoachLM: Automatic Instruction Revisions Improve the Data Quality in LLM Instruction Tuning](https://mlsanity.com/paper/2311.13246/) (2023)
- [Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval](https://mlsanity.com/paper/2105.03599/) (2021)
- [Data-Centric AI Requires Rethinking Data Notion](https://mlsanity.com/paper/2110.02491/) (2021)

## Built on by (most cited first, 30 of 53)

- [Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era](https://mlsanity.com/paper/2403.08946/) (2024)
- [I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm](https://mlsanity.com/paper/2408.08072/) (2024)
- [Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric](https://mlsanity.com/paper/2502.17184/) (2025)
- [D3: Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning](https://mlsanity.com/paper/2503.11441/) (2025)
- [Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation](https://mlsanity.com/paper/2508.10858/) (2025)
- [GATEAU: Selecting Influential Samples for Long Context Alignment](https://mlsanity.com/paper/2410.15633/) (2024)
- [CodeACT: Code Adaptive Compute-efficient Tuning Framework for Code LLMs](https://mlsanity.com/paper/2408.02193/) (2024)
- [Chasing Random: Instruction Selection Strategies Fail to Generalize](https://mlsanity.com/paper/2410.15225/) (2024)
- [Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models](https://mlsanity.com/paper/2406.14115/) (2024)
- [Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering](https://mlsanity.com/paper/2502.07340/) (2025)
- [Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencoder](https://mlsanity.com/paper/2502.14050/) (2025)
- [MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space](https://mlsanity.com/paper/2504.13835/) (2025)
- [API-guided Dataset Synthesis to Finetune Large Code Models](https://mlsanity.com/paper/2408.08343/) (2024)
- [Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge](https://mlsanity.com/paper/2412.01377/) (2024)
- [CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization](https://mlsanity.com/paper/2510.17847/) (2025)
- [Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning](https://mlsanity.com/paper/2505.17266/) (2025)
- [SageLM: A Multi-aspect and Explainable Large Language Model for Speech Judgement](https://mlsanity.com/paper/2508.20916/) (2025)
- [Baichuan Alignment Technical Report](https://mlsanity.com/paper/2410.14940/) (2024)
- [ELSPR: Evaluator LLM Training Data Self-Purification on Non-Transitive Preferences via Tournament Graph Reconstruction](https://mlsanity.com/paper/2505.17691/) (2025)
- [Retrieval Augmented Instruction Tuning for Open NER with Large Language Models](https://mlsanity.com/paper/2406.17305/) (2024)
- [DIDS: Domain Impact-aware Data Sampling for Large Language Model Training](https://mlsanity.com/paper/2504.13227/) (2025)
- [Annotation-Efficient Language Model Alignment via Diverse and Representative Response Texts](https://mlsanity.com/paper/2405.13541/) (2024)
- [MergeIT: From Selection to Merging for Efficient Instruction Tuning](https://mlsanity.com/paper/2503.00034/) (2025)
- [MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning](https://mlsanity.com/paper/2504.07288/) (2025)
- [RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selection](https://mlsanity.com/paper/2505.05327/) (2025)
- [From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning](https://mlsanity.com/paper/2505.24768/) (2025)
- [TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuning](https://mlsanity.com/paper/2510.07118/) (2025)
- [Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning](https://mlsanity.com/paper/2502.18978/) (2025)
- [MUSS: Multilevel Subset Selection for Relevance and Diversity](https://mlsanity.com/paper/2503.11126/) (2025)
- [GradientSpace: Unsupervised Data Clustering for Improved Instruction Tuning](https://mlsanity.com/paper/2512.06678/) (2025)

## Abstract (arXiv)

With contributions from the open-source community, a vast amount of instruction tuning (IT) data has emerged. Given the significant resource allocation required for training and evaluating models, it is advantageous to have an efficient method for selecting high-quality IT data. However, existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. In this paper, we propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). CaR employs a two-step process: first, it ranks instruction pairs using a high-accuracy (84.25%) scoring model aligned with expert preferences; second, it preserves dataset diversity through clustering. In our experiment, CaR efficiently selected a mere 1.96% of Alpaca's IT data, yet the resulting AlpaCaR model surpassed Alpaca's performance by an average of 32.1% in GPT-4 evaluations. Moreover, we find that data selecting is a consistent paradigm whether the pre-trained model is more capable or the model parameters scaling up. Our approach employs compact models with 550M parameters and incurs just 11.2% of the financial outlay of current methods, enhancing its industrial deployability.