# Contextualized Non-local Neural Networks for Sequence Learning

arXiv: https://arxiv.org/abs/1811.08600 · HTML: https://mlsanity.com/paper/1811.08600/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2018.

## Built on by (most cited first, 7 of 7)

- [Long-Range Transformers for Dynamic Spatiotemporal Forecasting](https://mlsanity.com/paper/2109.12218/) (2021)
- [GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling](https://mlsanity.com/paper/2106.01925/) (2021)
- [Data-centric Graph Learning: A Survey](https://mlsanity.com/paper/2310.04987/) (2023)
- [Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification](https://mlsanity.com/paper/2111.00180/) (2021)
- [Deep Unsupervised Active Learning on Learnable Graphs](https://mlsanity.com/paper/2111.04286/) (2021)
- [Progressive Fusion for Multimodal Integration](https://mlsanity.com/paper/2209.00302/) (2022)
- [Constructing Temporal Dynamic Knowledge Graphs from Interactive Text-based Games](https://mlsanity.com/paper/2311.01928/) (2023)

## Abstract (arXiv)

Recently, a large number of neural mechanisms and models have been proposed for sequence learning, of which self-attention, as exemplified by the Transformer model, and graph neural networks (GNNs) have attracted much attention. In this paper, we propose an approach that combines and draws on the complementary strengths of these two methods. Specifically, we propose contextualized non-local neural networks (CN3), which can both dynamically construct a task-specific structure of a sentence and leverage rich local dependencies within a particular neighborhood. Experimental results on ten NLP tasks in text classification, semantic matching, and sequence labeling show that our proposed model outperforms competitive baselines and discovers task-specific dependency structures, thus providing better interpretability to users.