# SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection

arXiv: https://arxiv.org/abs/2402.16705 · HTML: https://mlsanity.com/paper/2402.16705/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 51 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 30 of 43)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [LESS: Selecting Influential Data for Targeted Instruction Tuning](https://mlsanity.com/paper/2402.04333/) (2024)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor](https://mlsanity.com/paper/2212.09689/) (2022)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning](https://mlsanity.com/paper/2312.15685/) (2023)
- [Data Selection for Language Models via Importance Resampling](https://mlsanity.com/paper/2302.03169/) (2023)
- [Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision](https://mlsanity.com/paper/2305.03047/) (2023)
- [From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning](https://mlsanity.com/paper/2308.12032/) (2023)
- [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://mlsanity.com/paper/2311.10702/) (2023)
- [Scaling Neural Machine Translation](https://mlsanity.com/paper/1806.00187/) (2018)
- [Self-Alignment with Instruction Backtranslation](https://mlsanity.com/paper/2308.06259/) (2023)

## Built on by (most cited first, 30 of 34)

- [A Survey of Multimodal Large Language Model from A Data-centric Perspective](https://mlsanity.com/paper/2405.16640/) (2024)
- [DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory](https://mlsanity.com/paper/2410.08143/) (2024)
- [Rethinking Data Selection at Scale: Random Selection is Almost All You Need](https://mlsanity.com/paper/2410.09335/) (2024)
- [Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencoder](https://mlsanity.com/paper/2502.14050/) (2025)
- [MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space](https://mlsanity.com/paper/2504.13835/) (2025)
- [Only-IF:Revealing the Decisive Effect of Instruction Diversity on Generalization](https://mlsanity.com/paper/2410.04717/) (2024)
- [IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment](https://mlsanity.com/paper/2505.12762/) (2025)
- [SGIC: A Self-Guided Iterative Calibration Framework for RAG](https://mlsanity.com/paper/2506.16172/) (2025)
- [Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning](https://mlsanity.com/paper/2505.17266/) (2025)
- [SPICE: Submodular Penalized Information-Conflict Selection for Efficient Large Language Model Training](https://mlsanity.com/paper/2601.23155/) (2026)
- [Self-Powered LLM Modality Expansion for Large Speech-Text Models](https://mlsanity.com/paper/2410.03798/) (2024)
- [Too Easily Fooled? Prompt Injection Breaks LLMs on Frustratingly Simple Multiple-Choice Questions](https://mlsanity.com/paper/2508.13214/) (2025)
- [Data Selection for Multi-turn Dialogue Instruction Tuning](https://mlsanity.com/paper/2604.07892/) (2026)
- [InfoSFT: Learn More and Forget Less with Information-Aware Token Weighting](https://mlsanity.com/paper/2605.14967/) (2026)
- [RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selection](https://mlsanity.com/paper/2505.05327/) (2025)
- [Instruction Data Selection via Answer Divergence](https://mlsanity.com/paper/2604.10448/) (2026)
- [Tailoring the Curriculum: Student-Centered Reasoning Distillation via Dynamic Data-Model Compatibility](https://mlsanity.com/paper/2605.29229/) (2026)
- [From Symbolic Tasks to Code Generation: Diversification Yields Better Task Performers](https://mlsanity.com/paper/2405.19787/) (2024)
- [Data Quality Control in Federated Instruction-tuning of Large Language Models](https://mlsanity.com/paper/2410.11540/) (2024)
- [Boosting LLM via Learning from Data Iteratively and Selectively](https://mlsanity.com/paper/2412.17365/) (2024)
- [Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-Tuning](https://mlsanity.com/paper/2503.13383/) (2025)
- [LASER: Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy](https://mlsanity.com/paper/2505.22157/) (2025)
- [InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning](https://mlsanity.com/paper/2601.23006/) (2026)
- [On Representation Redundancy in Large-Scale Instruction Tuning Data Selection](https://mlsanity.com/paper/2602.13773/) (2026)
- [Neuron-Aware Data Selection In Instruction Tuning For Large Language Models](https://mlsanity.com/paper/2603.13201/) (2026)
- [ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework](https://mlsanity.com/paper/2604.07506/) (2026)
- [AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training](https://mlsanity.com/paper/2606.01635/) (2026)
- [Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Set](https://mlsanity.com/paper/2509.06463/) (2025)
- [CDT: A Comprehensive Capability Framework for Large Language Models Across Cognition, Domain, and Task](https://mlsanity.com/paper/2509.24422/) (2025)
- [From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuning](https://mlsanity.com/paper/2601.13995/) (2026)

## Abstract (arXiv)

Instruction tuning (IT) is crucial to tailoring large language models (LLMs) towards human-centric interactions. Recent advancements have shown that the careful selection of a small, high-quality subset of IT data can significantly enhance the performance of LLMs. Despite this, common approaches often rely on additional models or data, which increases costs and limits widespread adoption. In this work, we propose a novel approach, termed SelectIT, that capitalizes on the foundational capabilities of the LLM itself. Specifically, we exploit the intrinsic uncertainty present in LLMs to more effectively select high-quality IT data, without the need for extra resources. Furthermore, we introduce a curated IT dataset, the Selective Alpaca, created by applying SelectIT to the Alpaca-GPT4 dataset. Empirical results demonstrate that IT using Selective Alpaca leads to substantial model ability enhancement. The robustness of SelectIT has also been corroborated in various foundation models and domain-specific tasks. Our findings suggest that longer and more computationally intensive IT data may serve as superior sources of IT, offering valuable insights for future research in this area. Data, code, and scripts are freely available at https://github.com/Blue-Raincoat/SelectIT.