# Improving Task Diversity in Label Efficient Supervised Finetuning of LLMs

arXiv: https://arxiv.org/abs/2507.21482 · HTML: https://mlsanity.com/paper/2507.21482/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 20 of 20)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models](https://mlsanity.com/paper/2504.11468/) (2025)
- [Batch Active Learning at Scale](https://mlsanity.com/paper/2107.14263/) (2021)
- [MoDS: Model-oriented Data Selection for Instruction Tuning](https://mlsanity.com/paper/2311.15653/) (2023)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [Data Diversity Matters for Robust Instruction Tuning](https://mlsanity.com/paper/2311.14736/) (2023)
- [Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks](https://mlsanity.com/paper/2311.00288/) (2023)
- [SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models](https://mlsanity.com/paper/2403.07384/) (2024)
- [GALAXY: Graph-based Active Learning at the Extreme](https://mlsanity.com/paper/2202.01402/) (2022)
- [Diversity Measurement and Subset Selection for Instruction Tuning Datasets](https://mlsanity.com/paper/2402.02318/) (2024)
- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels](https://mlsanity.com/paper/2206.12569/) (2022)
- [LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning](https://mlsanity.com/paper/2306.09910/) (2023)
- [Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs](https://mlsanity.com/paper/2502.20356/) (2025)
- [Deep Active Learning in the Open World](https://mlsanity.com/paper/2411.06353/) (2024)

## Built on by (most cited first, 1 of 1)

- [Optimal Transport Depth Up-Scaling](https://mlsanity.com/paper/2508.08011/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, but developing high-performing models for specialized applications often requires substantial human annotation -- a process that is time-consuming, labor-intensive, and expensive. In this paper, we address the label-efficient learning problem for supervised finetuning (SFT) by leveraging task-diversity as a fundamental principle for effective data selection. This is markedly different from existing methods based on the prompt-diversity. Our approach is based on two key observations: 1) task labels for different prompts are often readily available; 2) pre-trained models have significantly varying levels of confidence across tasks. We combine these facts to devise a simple yet effective sampling strategy: we select examples across tasks using an inverse confidence weighting strategy. This produces models comparable to or better than those trained with more complex sampling procedures, while being significantly easier to implement and less computationally intensive. Notably, our experimental results demonstrate that this method can achieve better accuracy than training on the complete dataset (a 4% increase in MMLU score). Across various annotation budgets and two instruction finetuning datasets, our algorithm consistently performs at or above the level of the best existing methods, while reducing annotation costs by up to 80%.