# Portuguese Named Entity Recognition using BERT-CRF

arXiv: https://arxiv.org/abs/1909.10649 · HTML: https://mlsanity.com/paper/1909.10649/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2019.

## Built on by (most cited first, 30 of 73)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Spanish Pre-trained BERT Model and Evaluation Data](https://mlsanity.com/paper/2308.02976/) (2023)
- [SkillSpan: Hard and Soft Skill Extraction from English Job Postings](https://mlsanity.com/paper/2204.12811/) (2022)
- [Open, Closed, or Small Language Models for Text Classification?](https://mlsanity.com/paper/2308.10092/) (2023)
- [CySecBERT: A Domain-Adapted Language Model for the Cybersecurity Domain](https://mlsanity.com/paper/2212.02974/) (2022)
- [A systematic review of Hate Speech automatic detection using Natural Language Processing](https://mlsanity.com/paper/2106.00742/) (2021)
- [One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones](https://mlsanity.com/paper/2202.07028/) (2022)
- [Beyond Static Summarization: Proactive Memory Extraction for LLM Agents](https://mlsanity.com/paper/2601.04463/) (2026)
- [SemEval-2021 Task 11: NLPContributionGraph -- Structuring Scholarly NLP Contributions for a Research Knowledge Graph](https://mlsanity.com/paper/2106.07385/) (2021)
- [FindVehicle and VehicleFinder: A NER dataset for natural language-based vehicle retrieval and a keyword-based cross-modal vehicle retrieval system](https://mlsanity.com/paper/2304.10893/) (2023)
- [Conversational Recommender System and Large Language Model Are Made for Each Other in E-commerce Pre-sales Dialogue](https://mlsanity.com/paper/2310.14626/) (2023)
- [Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings](https://mlsanity.com/paper/2402.05617/) (2024)
- [E-NER -- An Annotated Named Entity Recognition Corpus of Legal Text](https://mlsanity.com/paper/2212.09306/) (2022)
- [Transformers: "The End of History" for NLP?](https://mlsanity.com/paper/2105.00813/) (2021)
- [De-identification of Privacy-related Entities in Job Postings](https://mlsanity.com/paper/2105.11223/) (2021)
- [U-NEED: A Fine-grained Dataset for User Needs-Centric E-commerce Conversational Recommendation](https://mlsanity.com/paper/2305.04774/) (2023)
- [BERTa'u: Ita'u BERT for digital customer service](https://mlsanity.com/paper/2101.12015/) (2021)
- [Casper: Prompt Sanitization for Protecting User Privacy in Web-Based Large Language Models](https://mlsanity.com/paper/2408.07004/) (2024)
- [Bertinho: Galician BERT Representations](https://mlsanity.com/paper/2103.13799/) (2021)
- [Multi-granularity Argument Mining in Legal Texts](https://mlsanity.com/paper/2210.09472/) (2022)
- [Introducing Rhetorical Parallelism Detection: A New Task with Datasets, Metrics, and Baselines](https://mlsanity.com/paper/2312.00100/) (2023)
- [NLRG at SemEval-2021 Task 5: Toxic Spans Detection Leveraging BERT-based Token Classification and Span Prediction Techniques](https://mlsanity.com/paper/2102.12254/) (2021)
- [WLV-RIT at SemEval-2021 Task 5: A Neural Transformer Framework for Detecting Toxic Spans](https://mlsanity.com/paper/2104.04630/) (2021)
- [German BERT Model for Legal Named Entity Recognition](https://mlsanity.com/paper/2303.05388/) (2023)
- [ReflectDiffu:Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Framework](https://mlsanity.com/paper/2409.10289/) (2024)
- [A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition](https://mlsanity.com/paper/2502.18702/) (2025)
- [UIUC_BioNLP at SemEval-2021 Task 11: A Cascade of Neural Models for Structuring Scholarly NLP Contributions](https://mlsanity.com/paper/2105.05435/) (2021)
- [Can BERT Dig It? -- Named Entity Recognition for Information Retrieval in the Archaeology Domain](https://mlsanity.com/paper/2106.07742/) (2021)
- [A novel knowledge graph development for industry design: A case study on indirect coal liquefaction process](https://mlsanity.com/paper/2111.13854/) (2021)
- [TMR: Evaluating NER Recall on Tough Mentions](https://mlsanity.com/paper/2103.12312/) (2021)

## Abstract (arXiv)

Recent advances in language representation using neural networks have made it viable to transfer the learned internal states of a trained model to downstream natural language processing tasks, such as named entity recognition (NER) and question answering. It has been shown that the leverage of pre-trained language models improves the overall performance on many tasks and is highly beneficial when labeled data is scarce. In this work, we train Portuguese BERT models and employ a BERT-CRF architecture to the NER task on the Portuguese language, combining the transfer capabilities of BERT with the structured predictions of CRF. We explore feature-based and fine-tuning training strategies for the BERT model. Our fine-tuning approach obtains new state-of-the-art results on the HAREM I dataset, improving the F1-score by 1 point on the selective scenario (5 NE classes) and by 4 points on the total scenario (10 NE classes).