# Task-Specific Dependency-based Word Embedding Methods

arXiv: https://arxiv.org/abs/2110.13376 · HTML: https://mlsanity.com/paper/2110.13376/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models](https://mlsanity.com/paper/2002.06652/) (2020)
- [Representation learning for very short texts using weighted word embedding aggregation](https://mlsanity.com/paper/1607.00570/) (2016)
- [Efficient Sentence Embedding via Semantic Subspace Analysis](https://mlsanity.com/paper/2002.09620/) (2020)

## Built on by (most cited first, 3 of 3)

- [Green Learning: Introduction, Examples and Outlook](https://mlsanity.com/paper/2210.00965/) (2022)
- [An Overview on Language Models: Recent Developments and Outlook](https://mlsanity.com/paper/2303.05759/) (2023)
- [SynWMD: Syntax-aware Word Mover's Distance for Sentence Similarity Evaluation](https://mlsanity.com/paper/2206.10029/) (2022)

## Abstract (arXiv)

Two task-specific dependency-based word embedding methods are proposed for text classification in this work. In contrast with universal word embedding methods that work for generic tasks, we design task-specific word embedding methods to offer better performance in a specific task. Our methods follow the PPMI matrix factorization framework and derive word contexts from the dependency parse tree. The first one, called the dependency-based word embedding (DWE), chooses keywords and neighbor words of a target word in the dependency parse tree as contexts to build the word-context matrix. The second method, named class-enhanced dependency-based word embedding (CEDWE), learns from word-context as well as word-class co-occurrence statistics. DWE and CEDWE are evaluated on popular text classification datasets to demonstrate their effectiveness. It is shown by experimental results they outperform several state-of-the-art word embedding methods.