mlsanityData to 14 Oct 2026

Using Language Models on Low-end Hardware

arXiv 2305.02350 ↗First posted 3 May 2023cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper evaluates the viability of using fixed language models for training text classification networks on low-end hardware. We combine language models with a CNN architecture and put together a comprehensive benchmark with 8 datasets covering single-label and multi-label classification of topic, sentiment, and genre. Our observations are distilled into a list of trade-offs, concluding that there are scenarios, where not fine-tuning a language model yields competitive effectiveness at faster training, requiring only a quarter of the memory compared to fine-tuning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

17 of 29 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  9. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  10. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  11. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  12. 2019To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Taskscited by 167
5 more
  1. 2021BertGCN: Transductive Text Classification by Combining GCN and BERTcited by 74
  2. 2017Multi-label Dataless Text Classification with Topic Modelingcited by 14
  3. 2019Hierarchical Attentional Hybrid Neural Networks for Document Classificationcited by 10
  4. 2022A Comparison of SVM against Pre-trained Language Models (PLMs) for Text Classification Taskscited by 7
  5. 2021Artificial Intelligence in the Global South (AI4D): Potential and Riskscited by 2

Built on by

Not cited yet.