mlsanityData to 14 Oct 2026

MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective

arXiv 2204.04391 ↗First posted 9 Apr 2022cs.CL cs.IRcited by 17

Not on the map. Why?

Abstract (arXiv)

NER model has achieved promising performance on standard NER benchmarks. However, recent studies show that previous approaches may over-rely on entity mention information, resulting in poor performance on out-of-vocabulary (OOV) entity recognition. In this work, we propose MINER, a novel NER learning framework, to remedy this issue from an information-theoretic perspective. The proposed approach contains two mutual information-based training objectives: i) generalizing information maximization, which enhances representation via deep understanding of context and entity surface forms; ii) superfluous information minimization, which discourages representation from rote memorizing entity names or exploiting biased cues in data. Experiments on various settings and datasets demonstrate that it achieves better performance in predicting OOV entities.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 32 of its 53 references that are in the map. First posted on arXiv 9 Apr 2022.

Builds on

32 of 53 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  7. 2018Deep contextualized word representationscited by 2,290
  8. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  9. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  10. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  11. 2016Deep Variational Information Bottleneckcited by 930
  12. 2016Neural Architectures for Named Entity Recognitioncited by 550
20 more
  1. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  2. 2015Named Entity Recognition with Bidirectional LSTM-CNNscited by 201
  3. 2020Learning Robust Representations via Multi-View Information Bottleneckcited by 191
  4. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  5. 2018Chinese NER Using Lattice LSTMcited by 81
  6. 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 76
  7. 2019TENER: Adapting Transformer Encoder for Named Entity Recognitioncited by 69
  8. 2021SpanNER: Named Entity Re-/Recognition as Span Predictioncited by 55
  9. 2021Joint Passage Ranking for Diverse Multi-Answer Retrievalcited by 43
  10. 2020Rethinking Generalization of Neural Models: A Named Entity Recognition Case Studycited by 35
  11. 2017Mimicking Word Embeddings using Subword RNNscited by 30
  12. 2019Few-Shot Representation Learning for Out-Of-Vocabulary Wordscited by 30
  13. 2020Coarse-to-Fine Pre-training for Named Entity Recognitioncited by 28
  14. 2020Multi-Domain Dialogue Acts and Response Co-Generationcited by 26
  15. 2020Named Entity Recognition for Social Media Texts with Semantic Augmentationcited by 23
  16. 2020Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improvecited by 16
  17. 2020The Dual Information Bottleneckcited by 14
  18. 2021Extracting Event Temporal Relations via Hyperbolic Geometrycited by 13
  19. 2017Robust Multilingual Named Entity Recognition with Shallow Semi-Supervised Featurescited by 8
  20. 2019Learning Task-specific Representation for Novel Words in Sequence Labelingcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 10; 2024: 3; 2025: 2.

17 papers.

Sort:

By area:

  1. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  2. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 170
  3. 2024Navigating the OverKill in Large Language Modelscited by 44
  4. 2023YAYI-UIE: A Chat-Enhanced Instruction Tuning Framework for Universal Information Extractioncited by 19
  5. 2023Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Modelcited by 12
  6. 2024LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertaintycited by 12
  7. 2022Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspectivecited by 10
  8. 2022PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Fillingcited by 9
  9. 2025VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Trainingcited by 9
  10. 2023E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognitioncited by 7
  11. 2023Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purificationcited by 4
  12. 2023Span-based Named Entity Recognition by Generating and Compressing Informationcited by 3
5 more
  1. 2023A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognitioncited by 3
  2. 2023Enhancing Phrase Representation by Information Bottleneck Guided Text Diffusion Process for Keyphrase Extractioncited by 2
  3. 2024RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructionscited by 2
  4. 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
  5. 2025Understanding and Mitigating Over-refusal for Large Language Models via Representation Interventioncited by 1