mlsanityData to 7 Oct 2026

Mitigating Negative Transfer with Task Awareness for Sexism, Hate Speech, and Toxic Language Detection

arXiv 2307.03377 ↗First posted 7 Jul 2023cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novelty approach to mitigate the negative transfer problem. In the field of machine learning, the common strategy is to apply the Single-Task Learning approach in order to train a supervised model to solve a specific task. Training a robust model requires a lot of data and a significant amount of computational resources, making this solution unfeasible in cases where data are unavailable or expensive to gather. Therefore another solution, based on the sharing of information between tasks, has been developed: Multi-Task Learning (MTL). Despite the recent developments regarding MTL, the problem of negative transfer has still to be solved. Negative transfer is a phenomenon that occurs when noisy information is shared between tasks, resulting in a drop in performance. This paper proposes a new approach to mitigate the negative transfer problem based on the task awareness concept. The proposed approach results in diminishing the negative transfer together with an improvement of performance over classic MTL solution. Moreover, the proposed approach has been implemented in two unified architectures to detect Sexism, Hate Speech, and Toxic Language in text comments. The proposed architectures set a new state-of-the-art both in EXIST-2021 and HatEval-2019 benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 54 references that are in the map. First posted on arXiv 7 Jul 2023.

Builds on

20 of 54 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  5. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  6. 2017A Survey on Multi-Task Learningcited by 838
  7. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  8. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  9. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  10. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  11. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 141
  12. 2020Understanding and Improving Information Transfer in Multi-Task Learningcited by 113
8 more
  1. 2016Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classificationcited by 93
  2. 2019Branched Multi-Task Networks: Deciding What Layers To Sharecited by 76
  3. 2018A Modulation Module for Multi-task Learning with Applications in Image Retrievalcited by 49
  4. 2020Multi-task learning for natural language processing in the 2020s: where are we going?cited by 27
  5. 2018Gradient Adversarial Training of Neural Networkscited by 17
  6. 2021Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Modelscited by 5
  7. 2022Compressed Hierarchical Representations for Multi-Task Learning and Task Clusteringcited by 2
  8. 2020Aggressive Language Detection with Joint Text Normalization via Adversarial Multi-task Learningcited by 1

Built on by

Not cited yet.