# Mix of Experts Language Model for Named Entity Recognition

arXiv: https://arxiv.org/abs/2404.19192 · HTML: https://mlsanity.com/paper/2404.19192/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision](https://mlsanity.com/paper/2006.15509/) (2020)
- [Mixture Models for Diverse Machine Translation: Tricks of the Trade](https://mlsanity.com/paper/1902.07816/) (2019)
- [Learning Named Entity Tagger using Domain-Specific Dictionary](https://mlsanity.com/paper/1809.03599/) (2018)
- [Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networks](https://mlsanity.com/paper/2311.10944/) (2023)
- [Exploring Diverse Methods in Visual Question Answering](https://mlsanity.com/paper/2404.13565/) (2024)
- [Low-Resource Name Tagging Learned with Weakly Labeled Data](https://mlsanity.com/paper/1908.09659/) (2019)
- [TFROM: A Two-sided Fairness-Aware Recommendation Model for Both Customers and Providers](https://mlsanity.com/paper/2104.09024/) (2021)
- [Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling](https://mlsanity.com/paper/1910.04289/) (2019)
- [A Comparative Study on Enhancing Prediction in Social Network Advertisement through Data Augmentation](https://mlsanity.com/paper/2404.13812/) (2024)
- [Few-shot Name Entity Recognition on StackOverflow](https://mlsanity.com/paper/2404.09405/) (2024)

## Built on by (most cited first, 11 of 11)

- [Enhance Image-to-Image Generation with LLaVA-generated Prompts](https://mlsanity.com/paper/2406.01956/) (2024)
- [Real-time Monitoring of Lower Limb Movement Resistance Based on Deep Learning](https://mlsanity.com/paper/2410.19769/) (2024)
- [LTPNet Integration of Deep Learning and Environmental Decision Support Systems for Renewable Energy Demand Forecasting](https://mlsanity.com/paper/2410.15286/) (2024)
- [Real-time Monitoring and Analysis of Track and Field Athletes Based on Edge Computing and Deep Reinforcement Learning Algorithm](https://mlsanity.com/paper/2411.06720/) (2024)
- [IoT-Based 3D Pose Estimation and Motion Optimization for Athletes: Application of C3D and OpenPose](https://mlsanity.com/paper/2411.12676/) (2024)
- [TRIZ Method for Urban Building Energy Optimization: GWO-SARIMA-LSTM Forecasting model](https://mlsanity.com/paper/2410.15283/) (2024)
- [Bias Evaluation and Mitigation in Retrieval-Augmented Medical Question-Answering Systems](https://mlsanity.com/paper/2503.15454/) (2025)
- [Optimized CNNs for Rapid 3D Point Cloud Object Recognition](https://mlsanity.com/paper/2412.02855/) (2024)
- [Intelligent logistics management robot path planning algorithm integrating transformer and GCN network](https://mlsanity.com/paper/2501.02749/) (2025)
- [GPTDrawer: Enhancing Visual Synthesis through ChatGPT](https://mlsanity.com/paper/2412.10429/) (2024)
- [Construction and optimization of health behavior prediction model for the elderly in smart elderly care](https://mlsanity.com/paper/2412.02062/) (2024)

## Abstract (arXiv)

Named Entity Recognition (NER) is an essential steppingstone in the field of natural language processing. Although promising performance has been achieved by various distantly supervised models, we argue that distant supervision inevitably introduces incomplete and noisy annotations, which may mislead the model training process. To address this issue, we propose a robust NER model named BOND-MoE based on Mixture of Experts (MoE). Instead of relying on a single model for NER prediction, multiple models are trained and ensembled under the Expectation-Maximization (EM) framework, so that noisy supervision can be dramatically alleviated. In addition, we introduce a fair assignment module to balance the document-model assignment process. Extensive experiments on real-world datasets show that the proposed method achieves state-of-the-art performance compared with other distantly supervised NER.