# Match-Ignition: Plugging PageRank into Transformer for Long-form Text Matching

arXiv: https://arxiv.org/abs/2101.06423 · HTML: https://mlsanity.com/paper/2101.06423/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 16 Jan 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [A Deep Relevance Matching Model for Ad-hoc Retrieval](https://mlsanity.com/paper/1711.08611/) (2017)
- [Bilateral Multi-Perspective Matching for Natural Language Sentences](https://mlsanity.com/paper/1702.03814/) (2017)
- [On Identifiability in Transformers](https://mlsanity.com/paper/1908.04211/) (2019)
- [Star-Transformer](https://mlsanity.com/paper/1902.09113/) (2019)
- [Convolutional Neural Network Architectures for Matching Natural Language Sentences](https://mlsanity.com/paper/1503.03244/) (2015)
- [A Deep Look into Neural Ranking Models for Information Retrieval](https://mlsanity.com/paper/1903.06902/) (2019)
- [Learning to Match Using Local and Distributed Representations of Text for Web Search](https://mlsanity.com/paper/1610.08136/) (2016)
- [Text Matching as Image Recognition](https://mlsanity.com/paper/1602.06359/) (2016)
- [Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval](https://mlsanity.com/paper/1502.06922/) (2015)
- [Content-Based Citation Recommendation](https://mlsanity.com/paper/1802.08301/) (2018)
- [A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations](https://mlsanity.com/paper/1511.08277/) (2015)
- [Simple and Effective Text Matching with Richer Alignment Features](https://mlsanity.com/paper/1908.00300/) (2019)
- [Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN](https://mlsanity.com/paper/1604.04378/) (2016)
- [PACRR: A Position-Aware Neural IR Model for Relevance Matching](https://mlsanity.com/paper/1704.03940/) (2017)
- [DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval](https://mlsanity.com/paper/1710.05649/) (2017)
- [Beyond Greedy Ranking: Slate Optimization via List-CVAE](https://mlsanity.com/paper/1803.01682/) (2018)
- [Modeling Diverse Relevance Patterns in Ad-hoc Retrieval](https://mlsanity.com/paper/1805.05737/) (2018)
- [Multilevel Text Alignment with Cross-Document Attention](https://mlsanity.com/paper/2010.01263/) (2020)
- [A Study of MatchPyramid Models on Ad-hoc Retrieval](https://mlsanity.com/paper/1606.04648/) (2016)
- [MatchZoo: A Toolkit for Deep Text Matching](https://mlsanity.com/paper/1707.07270/) (2017)

## Built on by (most cited first, 5 of 5)

- [RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling](https://mlsanity.com/paper/2310.10567/) (2023)
- [Match-Prompt: Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning](https://mlsanity.com/paper/2204.02725/) (2022)
- [Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents](https://mlsanity.com/paper/2203.02898/) (2022)
- [Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding](https://mlsanity.com/paper/2301.03765/) (2023)
- [CausalSent: Interpretable Sentiment Classification with RieszNet](https://mlsanity.com/paper/2508.17576/) (2025)

## Abstract (arXiv)

Neural text matching models have been widely used in community question answering, information retrieval, and dialogue. However, these models designed for short texts cannot well address the long-form text matching problem, because there are many contexts in long-form texts can not be directly aligned with each other, and it is difficult for existing models to capture the key matching signals from such noisy data. Besides, these models are computationally expensive for simply use all textual data indiscriminately. To tackle the effectiveness and efficiency problem, we propose a novel hierarchical noise filtering model, namely Match-Ignition. The main idea is to plug the well-known PageRank algorithm into the Transformer, to identify and filter both sentence and word level noisy information in the matching process. Noisy sentences are usually easy to detect because previous work has shown that their similarity can be explicitly evaluated by the word overlapping, so we directly use PageRank to filter such information based on a sentence similarity graph. Unlike sentences, words rely on their contexts to express concrete meanings, so we propose to jointly learn the filtering and matching process, to well capture the critical word-level matching signals. Specifically, a word graph is first built based on the attention scores in each self-attention block of Transformer, and key words are then selected by applying PageRank on this graph. In this way, noisy words will be filtered out layer by layer in the matching process. Experimental results show that Match-Ignition outperforms both SOTA short text matching models and recent long-form text matching models. We also conduct detailed analysis to show that Match-Ignition efficiently captures important sentences and words, to facilitate the long-form text matching process.