# Adaptable Text Matching via Meta-Weight Regulator

arXiv: https://arxiv.org/abs/2204.12668 · HTML: https://mlsanity.com/paper/2204.12668/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 27 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation](https://mlsanity.com/paper/2010.05862/) (2020)
- [Text Matching as Image Recognition](https://mlsanity.com/paper/1602.06359/) (2016)
- [Domain Adaptation as a Problem of Inference on Graphical Models](https://mlsanity.com/paper/2002.03278/) (2020)
- [Applying Deep Learning to Answer Selection: A Study and An Open Task](https://mlsanity.com/paper/1508.01585/) (2015)
- [Progressive Memory Banks for Incremental Domain Adaptation](https://mlsanity.com/paper/1811.00239/) (2018)
- [Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning](https://mlsanity.com/paper/1805.03403/) (2018)
- [Shortcut-Stacked Sentence Encoders for Multi-Domain Inference](https://mlsanity.com/paper/1708.02312/) (2017)
- [Improving Domain-Adapted Sentiment Classification by Deep Adversarial Mutual Learning](https://mlsanity.com/paper/2002.00119/) (2020)
- [Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching](https://mlsanity.com/paper/1812.11561/) (2018)

## Abstract (arXiv)

Neural text matching models have been used in a range of applications such as question answering and natural language inference, and have yielded a good performance. However, these neural models are of a limited adaptability, resulting in a decline in performance when encountering test examples from a different dataset or even a different task. The adaptability is particularly important in the few-shot setting: in many cases, there is only a limited amount of labeled data available for a target dataset or task, while we may have access to a richly labeled source dataset or task. However, adapting a model trained on the abundant source data to a few-shot target dataset or task is challenging. To tackle this challenge, we propose a Meta-Weight Regulator (MWR), which is a meta-learning approach that learns to assign weights to the source examples based on their relevance to the target loss. Specifically, MWR first trains the model on the uniformly weighted source examples, and measures the efficacy of the model on the target examples via a loss function. By iteratively performing a (meta) gradient descent, high-order gradients are propagated to the source examples. These gradients are then used to update the weights of source examples, in a way that is relevant to the target performance. As MWR is model-agnostic, it can be applied to any backbone neural model. Extensive experiments are conducted with various backbone text matching models, on four widely used datasets and two tasks. The results demonstrate that our proposed approach significantly outperforms a number of existing adaptation methods and effectively improves the cross-dataset and cross-task adaptability of the neural text matching models in the few-shot setting.