# Domain-Aware Contrastive Knowledge Transfer for Multi-domain Imbalanced Data

arXiv: https://arxiv.org/abs/2204.01916 · HTML: https://mlsanity.com/paper/2204.01916/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [Remix: Rebalanced Mixup](https://mlsanity.com/paper/2007.03943/) (2020)
- [Feature Space Augmentation for Long-Tailed Data](https://mlsanity.com/paper/2008.03673/) (2020)
- [M2m: Imbalanced Classification via Major-to-minor Translation](https://mlsanity.com/paper/2004.00431/) (2020)
- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective](https://mlsanity.com/paper/2002.10826/) (2020)
- [Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.05830/) (2015)
- [MDFEND: Multi-domain Fake News Detection](https://mlsanity.com/paper/2201.00987/) (2022)
- [Deep Imbalanced Attribute Classification using Visual Attention Aggregation](https://mlsanity.com/paper/1807.03903/) (2018)
- [Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks](https://mlsanity.com/paper/2112.03271/) (2021)
- [Survey of resampling techniques for improving classification performance in unbalanced datasets](https://mlsanity.com/paper/1608.06048/) (2016)
- [Learning Reporting Dynamics during Breaking News for Rumour Detection in Social Media](https://mlsanity.com/paper/1610.07363/) (2016)
- [Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents](https://mlsanity.com/paper/2010.11230/) (2020)

## Built on by (most cited first, 2 of 2)

- [Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification](https://mlsanity.com/paper/2311.16650/) (2023)
- [Scalable and Safe Remediation of Defective Actions in Self-Learning Conversational Systems](https://mlsanity.com/paper/2305.10528/) (2023)

## Abstract (arXiv)

In many real-world machine learning applications, samples belong to a set of domains e.g., for product reviews each review belongs to a product category. In this paper, we study multi-domain imbalanced learning (MIL), the scenario that there is imbalance not only in classes but also in domains. In the MIL setting, different domains exhibit different patterns and there is a varying degree of similarity and divergence among domains posing opportunities and challenges for transfer learning especially when faced with limited or insufficient training data. We propose a novel domain-aware contrastive knowledge transfer method called DCMI to (1) identify the shared domain knowledge to encourage positive transfer among similar domains (in particular from head domains to tail domains); (2) isolate the domain-specific knowledge to minimize the negative transfer from dissimilar domains. We evaluated the performance of DCMI on three different datasets showing significant improvements in different MIL scenarios.