# GWPT: A Green Word-Embedding-based POS Tagger

arXiv: https://arxiv.org/abs/2401.07475 · HTML: https://mlsanity.com/paper/2401.07475/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 15 Jan 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [Interpretable Convolutional Neural Networks via Feedforward Design](https://mlsanity.com/paper/1810.02786/) (2018)
- [PointHop: An Explainable Machine Learning Method for Point Cloud Classification](https://mlsanity.com/paper/1907.12766/) (2019)
- [PixelHop++: A Small Successive-Subspace-Learning-Based (SSL-based) Model for Image Classification](https://mlsanity.com/paper/2002.03141/) (2020)
- [Understanding Convolutional Neural Networks with A Mathematical Model](https://mlsanity.com/paper/1609.04112/) (2016)
- [DefakeHop: A Light-Weight High-Performance Deepfake Detector](https://mlsanity.com/paper/2103.06929/) (2021)
- [Green Learning: Introduction, Examples and Outlook](https://mlsanity.com/paper/2210.00965/) (2022)
- [Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation](https://mlsanity.com/paper/1508.02096/) (2015)
- [R-PointHop: A Green, Accurate, and Unsupervised Point Cloud Registration Method](https://mlsanity.com/paper/2103.08129/) (2021)
- [Hierarchically-Refined Label Attention Network for Sequence Labeling](https://mlsanity.com/paper/1908.08676/) (2019)
- [On Supervised Feature Selection from High Dimensional Feature Spaces](https://mlsanity.com/paper/2203.11924/) (2022)
- [On Data-Driven Saak Transform](https://mlsanity.com/paper/1710.04176/) (2017)
- [An Overview on Language Models: Recent Developments and Outlook](https://mlsanity.com/paper/2303.05759/) (2023)
- [Robust Multilingual Part-of-Speech Tagging via Adversarial Training](https://mlsanity.com/paper/1711.04903/) (2017)
- [Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings](https://mlsanity.com/paper/1805.08237/) (2018)
- [Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation](https://mlsanity.com/paper/1906.01569/) (2019)
- [Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network](https://mlsanity.com/paper/1510.06168/) (2015)
- [A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding](https://mlsanity.com/paper/1511.00215/) (2015)

## Abstract (arXiv)

As a fundamental tool for natural language processing (NLP), the part-of-speech (POS) tagger assigns the POS label to each word in a sentence. A novel lightweight POS tagger based on word embeddings is proposed and named GWPT (green word-embedding-based POS tagger) in this work. Following the green learning (GL) methodology, GWPT contains three modules in cascade: 1) representation learning, 2) feature learning, and 3) decision learning modules. The main novelty of GWPT lies in representation learning. It uses non-contextual or contextual word embeddings, partitions embedding dimension indices into low-, medium-, and high-frequency sets, and represents them with different N-grams. It is shown by experimental results that GWPT offers state-of-the-art accuracies with fewer model parameters and significantly lower computational complexity in both training and inference as compared with deep-learning-based methods.