mlsanityData to 7 Oct 2026

Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Models

arXiv 2412.11455 ↗First posted 16 Dec 2024cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

To efficiently select optimal dataset combinations for enhancing multi-task learning (MTL) performance in large language models, we proposed a novel framework that leverages a neural network to predict the best dataset combinations. The framework iteratively refines the selection, greatly improving efficiency, while being model-, dataset-, and domain-independent. Through experiments on 12 biomedical datasets across four tasks - named entity recognition, relation extraction, event extraction, and text classification-we demonstrate that our approach effectively identifies better combinations, even for tasks that may seem unpromising from a human perspective. This verifies that our framework provides a promising solution for maximizing MTL potential.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 21 of its 49 references that are in the map. First posted on arXiv 16 Dec 2024.

Builds on

21 of 49 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2017A Survey on Multi-Task Learningcited by 838
  4. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  5. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  6. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  7. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  8. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  9. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  10. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 167
  11. 2017PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstractscited by 123
  12. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 116
9 more
  1. 2022BioRED: A Rich Biomedical Relation Extraction Datasetcited by 84
  2. 2022Code4Struct: Code Generation for Few-Shot Event Structure Predictioncited by 70
  3. 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
  4. 2017Identifying beneficial task relations for multi-task learning in deep neural networkscited by 59
  5. 2023Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directionscited by 55
  6. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  7. 2024ADELIE: Aligning Large Language Models on Information Extractioncited by 24
  8. 2019AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learningcited by 22
  9. 2024RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplementscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learningcited by 13
  2. 2025An evaluation of DeepSeek Models in Biomedical Natural Language Processingcited by 6
  3. 2025Retrieval-augmented in-context learning for multimodal large language models in disease classificationcited by 4
  4. 2025Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendationcited by 3
  5. 2025EPEE: Towards Efficient and Effective Foundation Models in Biomedicinecited by 1
  6. 2026TRACE: Discovering Task-Specific Parameter via Adaptation-Aware Probing for Continual Fine-Tuningcited by 0