mlsanityData to 7 Oct 2026

MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces

arXiv 2109.04240 ↗First posted 9 Sep 2021cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Albeit the universal representational power of pre-trained language models, adapting them onto a specific NLP task still requires a considerably large amount of labeled data. Effective task fine-tuning meets challenges when only a few labeled examples are present for the task. In this paper, we aim to the address of the problem of few shot task learning by exploiting and transferring from a different task which admits a related but disparate label space. Specifically, we devise a label transfer network (LTN) to transform the labels from source task to the target task of interest for training. Both the LTN and the model for task prediction are learned via a bi-level optimization framework, which we term as MetaXT. MetaXT offers a principled solution to best adapt a pre-trained language model to the target task by transferring knowledge from the source task. Empirical evaluations on cross-task transfer settings for four NLP tasks, from two different types of label space disparities, demonstrate the effectiveness of MetaXT, especially when the labeled data in the target task is limited.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 9 Sep 2021.

Builds on

28 of 43 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  8. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  9. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  10. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  11. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  12. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
16 more
  1. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  2. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  3. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  4. 2018An introduction to domain adaptation and transfer learningcited by 112
  5. 2017Few-Shot Adversarial Domain Adaptationcited by 111
  6. 2019Meta Label Correction for Noisy Label Learningcited by 108
  7. 2017Adversarial Multi-task Learning for Text Classificationcited by 91
  8. 2020Few-shot Domain Adaptation by Causal Mechanism Transfercited by 47
  9. 2020Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake Newscited by 24
  10. 2021MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learningcited by 21
  11. 2018Meta Multi-Task Learning for Sequence Modelingcited by 18
  12. 2020Learning with Weak Supervision for Email Intent Detectioncited by 17
  13. 2018Multi-Task Learning for Sequence Tagging: An Empirical Studycited by 15
  14. 2017Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learningcited by 11
  15. 2018Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate Label Spacescited by 11
  16. 2017Multitask Learning for Fine-Grained Twitter Sentiment Analysiscited by 5

Built on by

Not cited yet.