Optimal Size-Performance Tradeoffs: Weighing PoS Tagger Models
Not on the map. Why?
Abstract (arXiv)
Improvement in machine learning-based NLP performance are often presented with bigger models and more complex code. This presents a trade-off: better scores come at the cost of larger tools; bigger models tend to require more during training and inference time. We present multiple methods for measuring the size of a model, and for comparing this with the model's performance. In a case study over part-of-speech tagging, we then apply these techniques to taggers for eight languages and present a novel analysis identifying which taggers are size-performance optimal. Results indicate that some classical taggers place on the size-performance skyline across languages. Further, although the deep models have highest performance for multiple scores, it is often not the most complex of these that reach peak performance.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 16 Apr 2021.
Builds on
24 of 35 references.
References · 24
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2016Bag of Tricks for Efficient Text Classificationcited by 869
- 2015Deep Learning with Limited Numerical Precisioncited by 368
- 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
- 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 163
- 2017BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languagescited by 52
- 2020A Comparison of LSTM and BERT for Small Corpuscited by 40
- 2016Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Losscited by 30
- 2017Robust Multilingual Part-of-Speech Tagging via Adversarial Trainingcited by 21
12 more
- 2013Polyglot: Distributed Word Representations for Multilingual NLPcited by 19
- 2018Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodingscited by 18
- 2020Controlling Information Capacity of Binary Neural Networkcited by 15
- 2019Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluationcited by 14
- 2018Near-lossless Binarization of Word Embeddingscited by 13
- 2020Embedding Compression with Isotropic Iterative Quantizationcited by 10
- 2016Cross-Lingual Morphological Tagging for Low-Resource Languagescited by 9
- 2018Learning Recurrent Binary/Ternary Weightscited by 8
- 2019A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languagescited by 7
- 2020Generation-Distillation for Efficient Natural Language Understanding in Low-Data Settingscited by 6
- 2016Compressing Neural Language Models by Sparse Word Representationscited by 4
- 2020Is the Best Better? Bayesian Statistical Model Comparison for Natural Language Processingcited by 4
Built on by
Citing papers per year: 2021: 1; 2022: 0; 2023: 0; 2024: 0; 2025: 1.
2 papers.