# NAT: Noise-Aware Training for Robust Neural Sequence Labeling

arXiv: https://arxiv.org/abs/2005.07162 · HTML: https://mlsanity.com/paper/2005.07162/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2020.

## Built on by (most cited first, 17 of 17)

- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Task](https://mlsanity.com/paper/2310.06504/) (2023)
- [TLDR9+: A Large Scale Resource for Extreme Summarization of Social Media Posts](https://mlsanity.com/paper/2110.01159/) (2021)
- [PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling](https://mlsanity.com/paper/2208.11508/) (2022)
- [Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labeling](https://mlsanity.com/paper/2105.11872/) (2021)
- [Revisit Out-Of-Vocabulary Problem for Slot Filling: A Unified Contrastive Frameword with Multi-level Data Augmentations](https://mlsanity.com/paper/2302.13584/) (2023)
- [Towards Robust and Generalizable Training: An Empirical Study of Noisy Slot Filling for Input Perturbations](https://mlsanity.com/paper/2310.03518/) (2023)
- [DemoNSF: A Multi-task Demonstration-based Generative Framework for Noisy Slot Filling Task](https://mlsanity.com/paper/2310.10169/) (2023)
- [Robust Learning for Text Classification with Multi-source Noise Simulation and Hard Example Mining](https://mlsanity.com/paper/2107.07113/) (2021)
- [DialAug: Mixing up Dialogue Contexts in Contrastive Learning for Robust Conversational Modeling](https://mlsanity.com/paper/2204.07679/) (2022)
- [Misspellings in Natural Language Processing: A survey](https://mlsanity.com/paper/2501.16836/) (2025)
- [PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation](https://mlsanity.com/paper/2201.02009/) (2022)
- [READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises](https://mlsanity.com/paper/2302.07324/) (2023)
- [Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation](https://mlsanity.com/paper/2407.18562/) (2024)
- [Noise-Aware Named Entity Recognition for Historical VET Documents](https://mlsanity.com/paper/2601.00488/) (2026)
- [Two-Step Occupation Coding](https://mlsanity.com/paper/2607.20101/) (2026)
- [ARID: A Deployable Edge AI System for Structured Information Extraction from Industrial Maintenance Work Orders](https://mlsanity.com/paper/2609.23582/) (2026)

## Abstract (arXiv)

Sequence labeling systems should perform reliably not only under ideal conditions but also with corrupted inputs - as these systems often process user-generated text or follow an error-prone upstream component. To this end, we formulate the noisy sequence labeling problem, where the input may undergo an unknown noising process and propose two Noise-Aware Training (NAT) objectives that improve robustness of sequence labeling performed on perturbed input: Our data augmentation method trains a neural model using a mixture of clean and noisy samples, whereas our stability training algorithm encourages the model to create a noise-invariant latent representation. We employ a vanilla noise model at training time. For evaluation, we use both the original data and its variants perturbed with real OCR errors and misspellings. Extensive experiments on English and German named entity recognition benchmarks confirmed that NAT consistently improved robustness of popular sequence labeling models, preserving accuracy on the original input. We make our code and data publicly available for the research community.