# Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings

arXiv: https://arxiv.org/abs/1805.08237 · HTML: https://mlsanity.com/paper/1805.08237/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2018.

## Built on by (most cited first, 18 of 18)

- [A Systematic Review of Reproducibility Research in Natural Language Processing](https://mlsanity.com/paper/2103.07929/) (2021)
- [Conformal prediction for text infilling and part-of-speech prediction](https://mlsanity.com/paper/2111.02592/) (2021)
- [MC-LSTM: Mass-Conserving LSTM](https://mlsanity.com/paper/2101.05186/) (2021)
- [On Leveraging Variational Graph Embeddings for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.11848/) (2022)
- [GNN-SL: Sequence Labeling Based on Nearest Examples via GNN](https://mlsanity.com/paper/2212.02017/) (2022)
- [Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling](https://mlsanity.com/paper/2101.11889/) (2021)
- [Locally-Contextual Nonlinear CRFs for Sequence Labeling](https://mlsanity.com/paper/2103.16210/) (2021)
- [Optimal Size-Performance Tradeoffs: Weighing PoS Tagger Models](https://mlsanity.com/paper/2104.07951/) (2021)
- [ELIT: Emory Language and Information Toolkit](https://mlsanity.com/paper/2109.03903/) (2021)
- [An Approach to Inference-Driven Dialogue Management within a Social Chatbot](https://mlsanity.com/paper/2111.00570/) (2021)
- [Minimal Supervision for Morphological Inflection](https://mlsanity.com/paper/2104.08512/) (2021)
- [Predictive Representation Learning for Language Modeling](https://mlsanity.com/paper/2105.14214/) (2021)
- [Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging](https://mlsanity.com/paper/2203.10315/) (2022)
- [Lite Unified Modeling for Discriminative Reading Comprehension](https://mlsanity.com/paper/2203.14103/) (2022)
- [Unleashing the True Potential of Sequence-to-Sequence Models for Sequence Tagging and Structure Parsing](https://mlsanity.com/paper/2302.02275/) (2023)
- [Multi-Task Learning for Front-End Text Processing in TTS](https://mlsanity.com/paper/2401.06321/) (2024)
- [GWPT: A Green Word-Embedding-based POS Tagger](https://mlsanity.com/paper/2401.07475/) (2024)
- [Benchmarking NLP-supported Language Sample Analysis for Swiss Children's Speech](https://mlsanity.com/paper/2504.00780/) (2025)

## Abstract (arXiv)

The rise of neural networks, and particularly recurrent neural networks, has produced significant advances in part-of-speech tagging accuracy. One characteristic common among these models is the presence of rich initial word encodings. These encodings typically are composed of a recurrent character-based representation with learned and pre-trained word embeddings. However, these encodings do not consider a context wider than a single word and it is only through subsequent recurrent layers that word or sub-word information interacts. In this paper, we investigate models that use recurrent neural networks with sentence-level context for initial character and word-based representations. In particular we show that optimal results are obtained by integrating these context sensitive representations through synchronized training with a meta-model that learns to combine their states. We present results on part-of-speech and morphological tagging with state-of-the-art performance on a number of languages.