mlsanityData to 14 Oct 2026

Semi-supervised sequence tagging with bidirectional language models

arXiv 1705.00108 ↗First posted 29 Apr 2017cs.CLcited by 86

Not on the map. Why?

Abstract (arXiv)

Pre-trained word embeddings learned from unlabeled text have become a standard component of neural network architectures for NLP tasks. However, in most cases, the recurrent network that operates on word-level representations to produce context sensitive representations is trained on relatively little labeled data. In this paper, we demonstrate a general semi-supervised approach for adding pre- trained context embeddings from bidirectional language models to NLP systems and apply it to sequence labeling tasks. We evaluate our model on two standard datasets for named entity recognition (NER) and chunking, and in both cases achieve state of the art results, surpassing previous systems that use other forms of transfer or joint learning with additional labeled data and task specific gazetteers.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 24; 2022: 22; 2023: 19; 2024: 11; 2025: 9.

86 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,234
  2. 2021A Survey on Deep Semi-supervised Learningcited by 251
  3. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  4. 2024Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modelingcited by 113
  5. 2021AR-LSAT: Investigating Analytical Reasoning of Textcited by 51
  6. 2022Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defensecited by 47
  7. 2021From LSAT: The Progress and Challenges of Complex Reasoningcited by 43
  8. 2022CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learningcited by 33
  9. 2023Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequencescited by 31
  10. 2022Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generationcited by 28
  11. 2021Should We Be Pre-training? An Argument for End-task Aware Training as an Alternativecited by 26
  12. 2024Position: Measure Dataset Diversity, Don't Just Claim Itcited by 26
38 more
  1. 2021Conformal prediction for text infilling and part-of-speech predictioncited by 22
  2. 2022There is a Time and Place for Reasoning Beyond the Imagecited by 22
  3. 2022Decomposed Meta-Learning for Few-Shot Named Entity Recognitioncited by 22
  4. 2021Neural ranking models for document retrievalcited by 16
  5. 2022DCT-Former: Efficient Self-Attention with Discrete Cosine Transformcited by 12
  6. 2022FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialoguecited by 11
  7. 2022Reasoning over Hybrid Chain for Table-and-Text Open Domain QAcited by 10
  8. 2023Recurrent Neural Networks and Long Short-Term Memory Networks: Tutorial and Surveycited by 10
  9. 2021Recent Trends in Named Entity Recognition (NER)cited by 9
  10. 2022Computer Science Named Entity Recognition in the Open Research Knowledge Graphcited by 9
  11. 2023A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NERcited by 9
  12. 2025Ensemble Learning for Large Language Models in Text and Code Generation: A Surveycited by 9
  13. 2022SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognitioncited by 8
  14. 2021Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviewscited by 7
  15. 2023Implicit Temporal Reasoning for Evidence-Based Fact-Checkingcited by 7
  16. 2023Several categories of Large Language Models (LLMs): A Short Surveycited by 7
  17. 2021Masked Conditional Random Fields for Sequence Labelingcited by 6
  18. 2023A Prototypical Semantic Decoupling Method via Joint Contrastive Learning for Few-Shot Name Entity Recognitioncited by 6
  19. 2023AcademicGPT: Empowering Academic Researchcited by 6
  20. 2025A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognitioncited by 6
  21. 2021Emotion Dynamics Modeling via BERTcited by 5
  22. 2021Theoretical foundations and limits of word embeddings: what types of meaning can they capture?cited by 5
  23. 2021Cross-Domain Generalization and Knowledge Transfer in Transformers Trained on Legal Datacited by 5
  24. 2022Downstream Transformer Generation of Question-Answer Pairs with Preprocessing and Postprocessing Pipelinescited by 5
  25. 2022Power of Explanations: Towards automatic debiasing in hate speech detectioncited by 5
  26. 2024SCANNER: Knowledge-Enhanced Approach for Robust Multi-modal Named Entity Recognition of Unseen Entitiescited by 5
  27. 2024FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generationcited by 5
  28. 2025RAMQA: A Unified Framework for Retrieval-Augmented Multi-Modal Question Answeringcited by 5
  29. 2025LOCORE: Image Re-ranking with Long-Context Sequence Modelingcited by 5
  30. 2021GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyphcited by 4
  31. 2021DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labelingcited by 4
  32. 2022The Lifecycle of "Facts": A Survey of Social Bias in Knowledge Graphscited by 4
  33. 2022TweetNERD -- End to End Entity Linking Benchmark for Tweetscited by 4
  34. 2023Uncertainty-aware Self-training for Low-resource Neural Sequence Labelingcited by 3
  35. 2023A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Modelscited by 3
  36. 2024RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-trainingcited by 3
  37. 2025VLMT: Vision-Language Multimodal Transformer for Multimodal Multi-hop Question Answeringcited by 3
  38. 2018Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Frameworkcited by 2

50 most cited of 86 All 86 (JSON)