mlsanityData to 7 Oct 2026

A Flexible Multi-Task Model for BERT Serving

arXiv 2107.05377 ↗First posted 12 Jul 2021cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In this demonstration, we present an efficient BERT-based multi-task (MT) framework that is particularly suitable for iterative and incremental development of the tasks. The proposed framework is based on the idea of partial fine-tuning, i.e. only fine-tune some top layers of BERT while keep the other layers frozen. For each task, we train independently a single-task (ST) model using partial fine-tuning. Then we compress the task-specific layers in each ST model using knowledge distillation. Those compressed ST models are finally merged into one MT model so that the frozen layers of the former are shared across the tasks. We exemplify our approach on eight GLUE tasks, demonstrating that it is able to achieve both strong performance and efficiency. We have implemented our method in the utterance understanding system of XiaoAI, a commercial AI assistant developed by Xiaomi. We estimate that our model reduces the overall serving cost by 86%.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

28 of 42 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2018Deep contextualized word representationscited by 2,285
  10. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  11. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  12. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
16 more
  1. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  2. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  3. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
  4. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  5. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  6. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  7. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  8. 2019To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Taskscited by 167
  9. 2020What Happens To BERT Embeddings During Fine-tuning?cited by 113
  10. 2020Understanding and Improving Information Transfer in Multi-Task Learningcited by 113
  11. 2020BERT-of-Theseus: Compressing BERT by Progressive Module Replacingcited by 112
  12. 2019Knowledge Distillation from Internal Representationscited by 87
  13. 2019BAM! Born-Again Multi-Task Networks for Natural Language Understandingcited by 85
  14. 2017Identifying beneficial task relations for multi-task learning in deep neural networkscited by 59
  15. 2020The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understandingcited by 28
  16. 2016When is multitask learning effective? Semantic sequence prediction under varying data conditionscited by 21

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 1.

4 papers.

  1. 2023On Optimal Caching and Model Multiplexing for Large Model Inferencecited by 20
  2. 2022Embedding Recycling for Language Modelscited by 5
  3. 2022All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Passcited by 0
  4. 2024Deploying Multi-task Online Server with Large Language Modelcited by 0