# Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching

arXiv: https://arxiv.org/abs/2001.07234 · HTML: https://mlsanity.com/paper/2001.07234/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2020.

## Built on by (most cited first, 5 of 5)

- [DABERT: Dual Attention Enhanced BERT for Semantic Matching](https://mlsanity.com/paper/2210.03454/) (2022)
- [Dual Path Modeling for Semantic Matching by Perceiving Subtle Conflicts](https://mlsanity.com/paper/2302.12530/) (2023)
- [Comateformer: Combined Attention Transformer for Semantic Sentence Matching](https://mlsanity.com/paper/2412.07220/) (2024)
- [CSSAM:Code Search via Attention Matching of Code Semantics and Structures](https://mlsanity.com/paper/2208.03922/) (2022)
- [Classical Sequence Match is a Competitive Few-Shot One-Class Learner](https://mlsanity.com/paper/2209.06394/) (2022)

## Abstract (arXiv)

Transformer has been successfully applied to many natural language processing tasks. However, for textual sequence matching, simple matching between the representation of a pair of sequences might bring in unnecessary noise. In this paper, we propose a new approach to sequence pair matching with Transformer, by learning head-wise matching representations on multiple levels. Experiments show that our proposed approach can achieve new state-of-the-art performance on multiple tasks that rely only on pre-computed sequence-vector-representation, such as SNLI, MNLI-match, MNLI-mismatch, QQP, and SQuAD-binary.