# FewTopNER: Integrating Few-Shot Learning with Topic Modeling and Named Entity Recognition in a Multilingual Framework

arXiv: https://arxiv.org/abs/2502.02391 · HTML: https://mlsanity.com/paper/2502.02391/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 4 Feb 2025.

## Builds on (most cited first, 27 of 27)

- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Cross-Lingual Ability of Multilingual BERT: An Empirical Study](https://mlsanity.com/paper/1912.07840/) (2019)
- [GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer](https://mlsanity.com/paper/2311.08526/) (2023)
- [Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification](https://mlsanity.com/paper/2205.02151/) (2022)
- [Few-shot classification in Named Entity Recognition Task](https://mlsanity.com/paper/1812.06158/) (2018)
- [CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2109.07589/) (2021)
- [Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages](https://mlsanity.com/paper/2010.03179/) (2020)
- [Decomposed Meta-Learning for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2204.05751/) (2022)
- [Example-Based Named Entity Recognition](https://mlsanity.com/paper/2008.10570/) (2020)
- [Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypes](https://mlsanity.com/paper/2208.08023/) (2022)
- [Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks](https://mlsanity.com/paper/2108.08375/) (2021)
- [Prompt-Based Metric Learning for Few-Shot NER](https://mlsanity.com/paper/2211.04337/) (2022)
- [NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval](https://mlsanity.com/paper/2310.14282/) (2023)
- [A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER](https://mlsanity.com/paper/2308.14533/) (2023)
- [SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition](https://mlsanity.com/paper/2210.09049/) (2022)
- [Improving Few-Shot Learning through Multi-task Representation Learning Theory](https://mlsanity.com/paper/2010.01992/) (2020)
- [LLM-RM at SemEval-2023 Task 2: Multilingual Complex NER using XLM-RoBERTa](https://mlsanity.com/paper/2305.03300/) (2023)
- [Comprehensive Overview of Named Entity Recognition: Models, Domain-Specific Applications and Challenges](https://mlsanity.com/paper/2309.14084/) (2023)
- [Few-shot Learning for Topic Modeling](https://mlsanity.com/paper/2104.09011/) (2021)
- [Multi-Level Contrastive Learning for Cross-Lingual Alignment](https://mlsanity.com/paper/2202.13083/) (2022)
- [ProtoNER: Few shot Incremental Learning for Named Entity Recognition using Prototypical Networks](https://mlsanity.com/paper/2310.02372/) (2023)
- [Large-Scale Label Interpretation Learning for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2403.14222/) (2024)
- [CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition](https://mlsanity.com/paper/2408.12834/) (2024)
- [Uppsala NLP at SemEval-2021 Task 2: Multilingual Language Models for Fine-tuning and Feature Extraction in Word-in-Context Disambiguation](https://mlsanity.com/paper/2104.03767/) (2021)
- [PALI at SemEval-2021 Task 2: Fine-Tune XLM-RoBERTa for Word in Context Disambiguation](https://mlsanity.com/paper/2104.10375/) (2021)
- [Cross-Level Cross-Scale Cross-Attention Network for Point Cloud Representation](https://mlsanity.com/paper/2104.13053/) (2021)
- [Multi-Task Learning of Query Intent and Named Entities using Transfer Learning](https://mlsanity.com/paper/2105.03316/) (2021)

## Built on by (most cited first, 1 of 1)

- [ANCHOLIK-NER: A Benchmark Dataset for Bangla Regional Named Entity Recognition](https://mlsanity.com/paper/2502.11198/) (2025)

## Abstract (arXiv)

We introduce FewTopNER, a novel framework that integrates few-shot named entity recognition (NER) with topic-aware contextual modeling to address the challenges of cross-lingual and low-resource scenarios. FewTopNER leverages a shared multilingual encoder based on XLM-RoBERTa, augmented with language-specific calibration mechanisms, to generate robust contextual embeddings. The architecture comprises a prototype-based entity recognition branch, employing BiLSTM and Conditional Random Fields for sequence labeling, and a topic modeling branch that extracts document-level semantic features through hybrid probabilistic and neural methods. A cross-task bridge facilitates dynamic bidirectional attention and feature fusion between entity and topic representations, thereby enhancing entity disambiguation by incorporating global semantic context. Empirical evaluations on multilingual benchmarks across English, French, Spanish, German, and Italian demonstrate that FewTopNER significantly outperforms existing state-of-the-art few-shot NER models. In particular, the framework achieves improvements of 2.5-4.0 percentage points in F1 score and exhibits enhanced topic coherence, as measured by normalized pointwise mutual information. Ablation studies further confirm the critical contributions of the shared encoder and cross-task integration mechanisms to the overall performance. These results underscore the efficacy of incorporating topic-aware context into few-shot NER and highlight the potential of FewTopNER for robust cross-lingual applications in low-resource settings.