# Dissecting Span Identification Tasks with Performance Prediction

arXiv: https://arxiv.org/abs/2010.02587 · HTML: https://mlsanity.com/paper/2010.02587/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 13 of 13)

- [Improving Adverse Drug Event Extraction with SpanBERT on Different Text Typologies](https://mlsanity.com/paper/2105.08882/) (2021)
- [PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks](https://mlsanity.com/paper/2210.08855/) (2022)
- [Developing a Named Entity Recognition Dataset for Tagalog](https://mlsanity.com/paper/2311.07161/) (2023)
- [Replicating and Extending "Because Their Treebanks Leak": Graph Isomorphism, Covariants, and Parser Performance](https://mlsanity.com/paper/2106.00352/) (2021)
- [Embarrassingly Simple Performance Prediction for Abductive Natural Language Inference](https://mlsanity.com/paper/2202.10408/) (2022)
- [Extensive Evaluation of Transformer-based Architectures for Adverse Drug Events Extraction](https://mlsanity.com/paper/2306.05276/) (2023)
- [UPB at SemEval-2021 Task 8: Extracting Semantic Information on Measurements as Multi-Turn Question Answering](https://mlsanity.com/paper/2104.04549/) (2021)
- [Span Identification of Epistemic Stance-Taking in Academic Written English](https://mlsanity.com/paper/2306.02038/) (2023)
- [IITK@Detox at SemEval-2021 Task 5: Semi-Supervised Learning and Dice Loss for Toxic Spans Detection](https://mlsanity.com/paper/2104.01566/) (2021)
- [UTNLP at SemEval-2021 Task 5: A Comparative Analysis of Toxic Span Detection using Attention-based, Named Entity Recognition, and Ensemble Models](https://mlsanity.com/paper/2104.04770/) (2021)
- [Meta Learning for Code Summarization](https://mlsanity.com/paper/2201.08310/) (2022)
- [Diagnosis extraction from unstructured Dutch echocardiogram reports using span- and document-level characteristic classification](https://mlsanity.com/paper/2408.06930/) (2024)
- [Towards a Diagnostic and Predictive Evaluation Methodology for Sequence Labeling Tasks](https://mlsanity.com/paper/2602.12759/) (2026)

## Abstract (arXiv)

Span identification (in short, span ID) tasks such as chunking, NER, or code-switching detection, ask models to identify and classify relevant spans in a text. Despite being a staple of NLP, and sharing a common structure, there is little insight on how these tasks' properties influence their difficulty, and thus little guidance on what model families work well on span ID tasks, and why. We analyze span ID tasks via performance prediction, estimating how well neural architectures do on different tasks. Our contributions are: (a) we identify key properties of span ID tasks that can inform performance prediction; (b) we carry out a large-scale experiment on English data, building a model to predict performance for unseen span ID tasks that can support architecture choices; (c), we investigate the parameters of the meta model, yielding new insights on how model and task properties interact to affect span ID performance. We find, e.g., that span frequency is especially important for LSTMs, and that CRFs help when spans are infrequent and boundaries non-distinctive.