Not Enough Labeled Data? Just Add Semantics: A Data-Efficient Method for Inferring Online Health Texts
Not on the map. Why?
Abstract (arXiv)
User-generated texts available on the web and social platforms are often long and semantically challenging, making them difficult to annotate. Obtaining human annotation becomes increasingly difficult as problem domains become more specialized. For example, many health NLP problems require domain experts to be a part of the annotation pipeline. Thus, it is crucial that we develop low-resource NLP solutions able to work with this set of limited-data problems. In this study, we employ Abstract Meaning Representation (AMR) graphs as a means to model low-resource Health NLP tasks sourced from various online health resources and communities. AMRs are well suited to model online health texts as they can represent multi-sentence inputs, abstract away from complex terminology, and model long-distance relationships between co-referring tokens. AMRs thus improve the ability of pre-trained language models to reason about high-complexity texts. Our experiments show that we can improve performance on 6 low-resource health NLP tasks by augmenting text embeddings with semantic graph embeddings. Our approach is task agnostic and easy to merge into any standard text classification pipeline. We experimentally validate that AMRs are useful in the modeling of complex texts by analyzing performance through the lens of two textual complexity measures: the Flesch Kincaid Reading Level and Syntactic Complexity. Our error analysis shows that AMR-infused language models perform better on complex texts and generally show less predictive variance in the presence of changing complexity.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 18 Sep 2023.
Builds on
28 of 53 references.
References · 28
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2015A large annotated corpus for learning natural language inferencecited by 1,837
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
- 2019Publicly Available Clinical BERT Embeddingscited by 995
- 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
- 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
16 more
- 2019Semantics-aware BERT for Language Understandingcited by 111
- 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 90
- 2020CoAID: COVID-19 Healthcare Misinformation Datasetcited by 83
- 2020Effective Transfer Learning for Identifying Similar Questions: Matching User Questions to COVID-19 FAQscited by 48
- 2020How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers?cited by 41
- 2020GPT-too: A language-model-first approach for AMR-to-text generationcited by 38
- 2019Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Setcited by 16
- 2021To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLPcited by 12
- 2022Public Wisdom Matters! Discourse-Aware Hyperbolic Fourier Co-Attention for Social-Text Classificationcited by 11
- 2022Monant Medical Misinformation Dataset: Mapping Articles to Fact-Checked Claimscited by 10
- 2021Data Augmentation for Mental Health Classification on Social Mediacited by 7
- 2022Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraintscited by 5
- 2022Scope of Pre-trained Language Models for Detecting Conflicting Health Informationcited by 4
- 2022Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representationcited by 4
- 2022Semantic-based Pre-training for Dialogue Understandingcited by 3
- 2022HealthE: Classifying Entities in Online Textual Health Advicecited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.