mlsanityData to 14 Oct 2026

Bringing Emerging Architectures to Sequence Labeling in NLP

arXiv 2509.25918 ↗First posted 30 Sep 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Pretrained Transformer encoders are the dominant approach to sequence labeling. While some alternative architectures-such as xLSTMs, structured state-space models, diffusion models, and adversarial learning-have shown promise in language modeling, few have been applied to sequence labeling, and mostly on flat or simplified tasks. We study how these architectures adapt across tagging tasks that vary in structural complexity, label space, and token dependencies, with evaluation spanning multiple languages. We find that the strong performance previously observed in simpler settings does not always generalize well across languages or datasets, nor does it extend to more complex structured tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 30 of its 75 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

30 of 75 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  6. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  7. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 376
  8. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  9. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 271
  10. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  11. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  12. 2018Multilingual Constituency Parsing with Self-Attention and Pre-Trainingcited by 117
18 more
  1. 2021KLUE: Korean Language Understanding Evaluationcited by 114
  2. 2018Linguistically-Informed Self-Attention for Semantic Role Labelingcited by 71
  3. 2016GANS for Sequences of Discrete Elements with the Gumbel-softmax Distributioncited by 48
  4. 2022Empowering Diffusion Models on the Embedding Space for Text Generationcited by 40
  5. 2019Viable Dependency Parsing as Sequence Labelingcited by 34
  6. 2018Constituent Parsing as Sequence Labelingcited by 26
  7. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generationcited by 21
  8. 2019Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inferencecited by 17
  9. 2022HiNER: A Large Hindi Named Entity Recognition Datasetcited by 12
  10. 2020Discontinuous Constituent Parsing as Sequence Labelingcited by 9
  11. 20234 and 7-bit Labeling for Projective and Non-Projective Dependency Treescited by 8
  12. 2023A Diffusion Weighted Graph Framework for New Intent Discoverycited by 7
  13. 2020Inherent Dependency Displacement Bias of Transition-Based Algorithmscited by 4
  14. 2025MolGraph-xLSTM: A graph-based dual-level xLSTM framework with multi-head mixture-of-experts for enhanced molecular representation and interpretabilitycited by 3
  15. 2023On the Challenges of Fully Incremental Neural Dependency Parsingcited by 2
  16. 2024Dependency Graph Parsing as Sequence Labelingcited by 2
  17. 2025A Study on the Performance of U-Net Modifications in Retroperitoneal Tumor Segmentationcited by 2
  18. 2024From Partial to Strictly Incremental Constituent Parsingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Toward Federated Large Language Models in Medicine: A Parameter-Efficient Framework for Privacy-Preserving, Multi-Institutional Adaptationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2021Accelerating BERT Inference for Sequence Labeling via Early-Exit
  2. 2022Efficient Long Sequence Encoding via Synchronization
  3. 2024Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performance
  4. 2025Hierarchical Bracketing Encodings for Dependency Parsing as Tagging
  5. 2025TiME: Tiny Monolingual Encoders for Efficient NLP Pipelines
  6. 2022Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction
  7. 2026DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models
  8. 2020Aligning the Pretraining and Finetuning Objectives of Language Models