mlsanityData to 14 Oct 2026

DABERT: Dual Attention Enhanced BERT for Semantic Matching

arXiv 2210.03454 ↗First posted 7 Oct 2022cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Transformer-based pre-trained language models such as BERT have achieved remarkable results in Semantic Sentence Matching. However, existing models still suffer from insufficient ability to capture subtle differences. Minor noise like word addition, deletion, and modification of sentences may cause flipped predictions. To alleviate this problem, we propose a novel Dual Attention Enhanced BERT (DABERT) to enhance the ability of BERT to capture fine-grained differences in sentence pairs. DABERT comprises (1) Dual Attention module, which measures soft word matches by introducing a new dual channel alignment mechanism to model affinity and difference attention. (2) Adaptive Fusion module, this module uses attention to learn the aggregation of difference and affinity features, and generates a vector describing the matching details of sentence pairs. We conduct extensive experiments on well-studied semantic matching and robustness test datasets, and the experimental results show the effectiveness of our proposed method.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 7 Oct 2022.

Builds on

24 of 38 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  10. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 432
  11. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
  12. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 304
12 more
  1. 2016Enhanced LSTM for Natural Language Inferencecited by 200
  2. 2017Bilateral Multi-Perspective Matching for Natural Language Sentencescited by 121
  3. 2019Semantics-aware BERT for Language Understandingcited by 112
  4. 2017A Continuously Growing Dataset of Sentential Paraphrasescited by 72
  5. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 56
  6. 2021Syntax-BERT: Improving Pre-trained Transformers with Syntax Treescited by 51
  7. 2021ExplainaBoard: An Explainable Leaderboard for NLPcited by 50
  8. 2017Learning to Compose Task-Specific Tree Structurescited by 48
  9. 2015A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representationscited by 32
  10. 2021Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Taskscited by 22
  11. 2018Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequencescited by 10
  12. 2020Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matchingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 3; 2025: 7; 2026: 8.

21 papers.

Sort:

By area:

  1. 2024Elite360D: Towards Efficient 360 Depth Estimation via Semantic- and Distance-Aware Bi-Projection Fusioncited by 31
  2. 2023Time-aware Multiway Adaptive Fusion Network for Temporal Knowledge Graph Question Answeringcited by 27
  3. 2023Dual Path Modeling for Semantic Matching by Perceiving Subtle Conflictscited by 21
  4. 2022Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusioncited by 18
  5. 2025Not All Parameters Are Created Equal: Smart Isolation Boosts Fine-Tuning Performancecited by 13
  6. 2025Structural Reward Model: Enhancing Interpretability, Efficiency, and Scalability in Reward Modelingcited by 11
  7. 2024Comateformer: Combined Attention Transformer for Semantic Sentence Matchingcited by 9
  8. 2025StructCoh: Structured Contrastive Learning for Context-Aware Text Semantic Matchingcited by 8
  9. 2026Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Modelscited by 8
  10. 2025DeCoRL: Decoupling Reasoning Chains via Parallel Sub-Step Generation and Cascaded Reinforcement for Interpretable and Scalable RLHFcited by 7
  11. 2026Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertaintycited by 6
  12. 2026DPI: Exploiting Parameter Heterogeneity for Interference-Free Fine-Tuningcited by 5
9 more
  1. 2024Elite360M: Efficient 360 Multi-task Learning via Bi-projection Fusion and Cross-task Collaborationcited by 3
  2. 2026Reinforcement Learning Enhanced Multi-hop Reasoning for Temporal Knowledge Question Answeringcited by 2
  3. 2026Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuningcited by 2
  4. 2025Using External knowledge to Enhanced PLM for Semantic Matchingcited by 0
  5. 2025Boosting Neural Language Inference via Cascaded Interactive Reasoningcited by 0
  6. 2025Glass Surface Detection: Leveraging Reflection Dynamics in Flash/No-flash Imagerycited by 0
  7. 2026Temporal-Aware Heterogeneous Graph Reasoning with Multi-View Fusion for Temporal Question Answeringcited by 0
  8. 2026Efficient Task Adaptation in Large Language Models via Selective Parameter Optimizationcited by 0
  9. 2026Multi-Granularity Reasoning for Natural Language Inferencecited by 0