# PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks

arXiv: https://arxiv.org/abs/1508.00200 · HTML: https://mlsanity.com/paper/1508.00200/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2015.

## Built on by (most cited first, 30 of 96)

- [Graph Learning: A Survey](https://mlsanity.com/paper/2105.00696/) (2021)
- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [Graph-based Semi-supervised Learning: A Comprehensive Review](https://mlsanity.com/paper/2102.13303/) (2021)
- [BertGCN: Transductive Text Classification by Combining GCN and BERT](https://mlsanity.com/paper/2105.05727/) (2021)
- [A Comprehensive Survey on Deep Graph Representation Learning](https://mlsanity.com/paper/2304.05055/) (2023)
- [TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter](https://mlsanity.com/paper/2209.07562/) (2022)
- [HINormer: Representation Learning On Heterogeneous Information Networks with Graph Transformer](https://mlsanity.com/paper/2302.11329/) (2023)
- [A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications](https://mlsanity.com/paper/2202.07893/) (2022)
- [MATCH: Metadata-Aware Text Classification in A Large Hierarchy](https://mlsanity.com/paper/2102.07349/) (2021)
- [Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding](https://mlsanity.com/paper/2110.07582/) (2021)
- [Beyond Low-Pass Filters: Adaptive Feature Propagation on Graphs](https://mlsanity.com/paper/2103.14187/) (2021)
- [Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP](https://mlsanity.com/paper/2109.03777/) (2021)
- [Graph Neural Networks: Methods, Applications, and Opportunities](https://mlsanity.com/paper/2108.10733/) (2021)
- [Heterogeneous Graph Contrastive Multi-view Learning](https://mlsanity.com/paper/2210.00248/) (2022)
- [Multiplex Bipartite Network Embedding using Dual Hypergraph Convolutional Networks](https://mlsanity.com/paper/2102.06371/) (2021)
- [Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data](https://mlsanity.com/paper/2109.10856/) (2021)
- [Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification](https://mlsanity.com/paper/2111.00180/) (2021)
- [CogDL: A Comprehensive Library for Graph Deep Learning](https://mlsanity.com/paper/2103.00959/) (2021)
- [Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts](https://mlsanity.com/paper/2212.06002/) (2022)
- [Enhancing Interpretable Clauses Semantically using Pretrained Word Representation](https://mlsanity.com/paper/2104.06901/) (2021)
- [ME-GCN: Multi-dimensional Edge-Embedded Graph Convolutional Networks for Semi-supervised Text Classification](https://mlsanity.com/paper/2204.04618/) (2022)
- [Embedding Heterogeneous Networks into Hyperbolic Space Without Meta-path](https://mlsanity.com/paper/2106.09923/) (2021)
- [REFINE: Random RangE FInder for Network Embedding](https://mlsanity.com/paper/2108.10703/) (2021)
- [MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information](https://mlsanity.com/paper/2111.04022/) (2021)
- [Semantic-aware Node Synthesis for Imbalanced Heterogeneous Information Networks](https://mlsanity.com/paper/2302.14061/) (2023)
- [GraphAttacker: A General Multi-Task GraphAttack Framework](https://mlsanity.com/paper/2101.06855/) (2021)
- [A Robust and Generalized Framework for Adversarial Graph Embedding](https://mlsanity.com/paper/2105.10651/) (2021)
- [InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification](https://mlsanity.com/paper/2109.07319/) (2021)
- [Recommender systems based on graph embedding techniques: A comprehensive review](https://mlsanity.com/paper/2109.09587/) (2021)
- [Weakly Supervised Learning for Analyzing Political Campaigns on Facebook](https://mlsanity.com/paper/2210.10669/) (2022)

## Abstract (arXiv)

Unsupervised text embedding methods, such as Skip-gram and Paragraph Vector, have been attracting increasing attention due to their simplicity, scalability, and effectiveness. However, comparing to sophisticated deep learning architectures such as convolutional neural networks, these methods usually yield inferior results when applied to particular machine learning tasks. One possible reason is that these text embedding methods learn the representation of text in a fully unsupervised way, without leveraging the labeled information available for the task. Although the low dimensional representations learned are applicable to many different tasks, they are not particularly tuned for any task. In this paper, we fill this gap by proposing a semi-supervised representation learning method for text data, which we call the textitpredictive text embedding (PTE). Predictive text embedding utilizes both labeled and unlabeled data to learn the embedding of text. The labeled information and different levels of word co-occurrence information are first represented as a large-scale heterogeneous text network, which is then embedded into a low dimensional space through a principled and efficient algorithm. This low dimensional embedding not only preserves the semantic closeness of words and documents, but also has a strong predictive power for the particular task. Compared to recent supervised approaches based on convolutional neural networks, predictive text embedding is comparable or more effective, much more efficient, and has fewer parameters to tune.