Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens
Not on the map. Why?
Abstract (arXiv)
Can attention- or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels? We construct a neural network architecture based on soft attention, train it as a binary sentence classifier and evaluate against token-level annotation on four different datasets. Inferring token labels from a network provides a method for quantitatively evaluating what the model is learning, along with generating useful feedback in assistance systems. Our results indicate that attention-based methods are able to predict token-level labels more accurately, compared to gradient-based methods, sometimes even rivaling the supervised oracle network.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 May 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 5; 2023: 3; 2024: 1.
12 papers.
- 2022On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliencycited by 31
- 2021Supervising Model Attention with Human Explanations for Robust Natural Language Inferencecited by 29
- 2022Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifoldcited by 29
- 2022Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Modelscited by 18
- 2023Atomic Inference for NLI with Generated Facts as Atomscited by 18
- 2022Few-shot Named Entity Recognition with Self-describing Networkscited by 16
- 2021Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Frameworkcited by 15
- 2021Zero-shot Sequence Labeling for Transformer-based Sentence Classifierscited by 5
- 2023Tell Model Where to Attend: Improving Interpretability of Aspect-Based Sentiment Classification via Small Explanation Annotationscited by 2
- 2022Utilizing coarse-grained data in low-data settings for event extractioncited by 1
- 2023Finding the Needle in a Haystack: Unsupervised Rationale Extraction from Long Text Classifierscited by 1
- 2024Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysiscited by 1
- 2024Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysis30 Sep 2024
- 2023Atomic Inference for NLI with Generated Facts as Atoms22 May 2023
- 2023Finding the Needle in a Haystack: Unsupervised Rationale Extraction from Long Text Classifiers14 Mar 2023
- 2023Tell Model Where to Attend: Improving Interpretability of Aspect-Based Sentiment Classification via Small Explanation Annotations21 Feb 2023
- 2022Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold20 Jun 2022
- 2022Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models23 May 2022
- 2022Utilizing coarse-grained data in low-data settings for event extraction11 May 2022
- 2022Few-shot Named Entity Recognition with Self-describing Networks23 Mar 2022
- 2022On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliency14 Mar 2022
- 2021Learning from Language Description: Low-shot Named Entity Recognition via Decomposed Framework11 Sep 2021
- 2021Supervising Model Attention with Human Explanations for Robust Natural Language Inference16 Apr 2021
- 2021Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers26 Mar 2021