mlsanityData to 14 Oct 2026

'UFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5

arXiv 2110.15248 ↗First posted 28 Oct 2021cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

We present the winning entry to the Multilingual Lexical Normalization (MultiLexNorm) shared task at W-NUT 2021 (van der Goot et al., 2021a), which evaluates lexical-normalization systems on 12 social media datasets in 11 languages. We base our solution on a pre-trained byte-level language model, ByT5 (Xue et al., 2021a), which we further pre-train on synthetic data and then fine-tune on authentic normalization data. Our system achieves the best performance by a wide margin in intrinsic evaluation, and also the best performance in extrinsic evaluation through dependency parsing. The source code is released at https://github.com/ufal/multilexnorm2021 and the fine-tuned models at https://huggingface.co/ufal.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 18 of its 39 references that are in the map. First posted on arXiv 28 Oct 2021.

Builds on

18 of 39 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  6. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  7. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  8. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  9. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  10. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  11. 2021A Simple Recipe for Multilingual Grammatical Error Correctioncited by 108
  12. 2019Parallel Iterative Edit Models for Local Sequence Transductioncited by 81
6 more
  1. 2019TWEETQA: A Social Media Focused Question Answering Datasetcited by 55
  2. 2020Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLPcited by 50
  3. 2021DaN+: Danish Nested Named Entities and Lexical Normalizationcited by 19
  4. 2019Adapting Sequence to Sequence models for Text Normalization in Social Mediacited by 8
  5. 2020Lexical Normalization for Code-switched Data and its Effect on POS-taggingcited by 7
  6. 2021Sequence-to-Sequence Lexical Normalization with Multilingual Transformerscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 0; 2024: 3; 2025: 1; 2026: 2.

7 papers.

  1. 2022Correcting diacritics and typos with a ByT5 transformer modelcited by 6
  2. 2024Neural Text Normalization for Luxembourgish using Real-Life Variation Datacited by 6
  3. 2024Making Sentence Embeddings Robust to User-Generated Contentcited by 1
  4. 2025Comprehensive Evaluation on Lexical Normalization: Boundary-Aware Approaches for Unsegmented Languagescited by 1
  5. 2026MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languagescited by 1
  6. 2024Identifying and Aligning Medical Claims Made on Social Media with Medical Evidencecited by 0
  7. 2026The Curse of Multilinguality in Lexical Normalizationcited by 0