mlsanityData to 7 Oct 2026

Adaptor: Objective-Centric Adaptation Framework for Language Models

arXiv 2203.03989 ↗First posted 8 Mar 2022cs.CL cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Progress in natural language processing research is catalyzed by the possibilities given by the widespread software frameworks. This paper introduces Adaptor library that transposes the traditional model-centric approach composed of pre-training + fine-tuning steps to objective-centric approach, composing the training process by applications of selected objectives. We survey research directions that can benefit from enhanced objective-centric experimentation in multitask training, custom objectives development, dynamic training curricula, or domain adaptation. Adaptor aims to ease reproducibility of these research directions in practice. Finally, we demonstrate the practical applicability of Adaptor in selected unsupervised domain adaptation scenarios.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 8 Mar 2022.

Builds on

28 of 43 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  6. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  9. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  10. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  11. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  12. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
16 more
  1. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  2. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  3. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  4. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  5. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  6. 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 259
  7. 2018Marian: Fast Neural Machine Translation in C++cited by 202
  8. 2020Adversarial Filters of Dataset Biasescited by 159
  9. 2020An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Modelscited by 133
  10. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  11. 2019Bridging the Gap between Training and Inference for Neural Machine Translationcited by 123
  12. 2020Towards Debiasing NLU Models from Unknown Biasescited by 116
  13. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  14. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  15. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  16. 2020Improving QA Generalization by Concurrent Modeling of Multiple Biasescited by 12

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 4; 2024: 2.

7 papers.

  1. 2023Resources and Few-shot Learners for In-context Learning in Slavic Languagescited by 6
  2. 2023Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Modelscited by 6
  3. 2024Concept-aware Data Construction Improves In-context Learning of Language Modelscited by 2
  4. 2024Multi-Task Domain Adaptation for Language Grounding with 3D Objectscited by 2
  5. 2022Soft Alignment Objectives for Robust Adaptation of Language Generationcited by 0
  6. 2023Concept-aware Training Improves In-context Learning Ability of Language Modelscited by 0
  7. 2023People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Textscited by 0