mlsanityData to 7 Oct 2026

Mutual Information Maximization for Simple and Accurate Part-Of-Speech Induction

arXiv 1804.07849 ↗First posted 20 Apr 2018cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

We address part-of-speech (POS) induction by maximizing the mutual information between the induced label and its context. We focus on two training objectives that are amenable to stochastic gradient descent (SGD): a novel generalization of the classical Brown clustering objective and a recently proposed variational lower bound. While both objectives are subject to noise in gradient updates, we show through analysis and experiments that the variational lower bound is robust whereas the generalized Brown objective is vulnerable. We obtain competitive performance on a multitude of datasets and languages with a simple architecture that encodes morphology and context.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 7; 2023: 2; 2024: 2; 2025: 2; 2026: 1.

17 papers.

Sort:

By area:

  1. 2022An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labelscited by 88
  2. 2023MINDE: Mutual Information Neural Diffusion Estimationcited by 31
  3. 2021Decomposed Mutual Information Estimation for Contrastive Representation Learningcited by 24
  4. 2022MVP: Multi-task Supervised Pre-training for Natural Language Generationcited by 16
  5. 2022Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionalitycited by 8
  6. 2024Information Theoretic Text-to-Image Alignmentcited by 8
  7. 2021CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervisioncited by 7
  8. 2025InfoBridge: Mutual Information estimation via Bridge Matchingcited by 7
  9. 2025INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimatorscited by 2
  10. 2022Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Taggingcited by 1
  11. 2022MICO: Selective Search with Mutual Information Co-trainingcited by 1
  12. 2024Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddingscited by 1
5 more
  1. 2021On planetary systems as ordered sequencescited by 0
  2. 2022Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging?cited by 0
  3. 2022Decomposed Mutual Information Optimization for Generalized Context in Meta-Reinforcement Learningcited by 0
  4. 2023Review of Unsupervised POS Tagging and Its Implications on Language Acquisitioncited by 0
  5. 2026ALICE: In-context, Zero-shot, Mutual Information Estimationcited by 0