mlsanityData to 14 Oct 2026

Multi-Attribute Multi-Grained Adaptation of Pre-Trained Language Models for Text Understanding from Bayesian Perspective

arXiv 2503.06085 ↗First posted 8 Mar 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Current neural networks often employ multi-domain-learning or attribute-injecting mechanisms to incorporate non-independent and identically distributed (non-IID) information for text understanding tasks by capturing individual characteristics and the relationships among samples. However, the extent of the impact of non-IID information and how these methods affect pre-trained language models (PLMs) remains unclear. This study revisits the assumption that non-IID information enhances PLMs to achieve performance improvements from a Bayesian perspective, which unearths and integrates non-IID and IID features. Furthermore, we proposed a multi-attribute multi-grained framework for PLM adaptations (M2A), which combines multi-attribute and multi-grained views to mitigate uncertainty in a lightweight manner. We evaluate M2A through prevalent text-understanding datasets and demonstrate its superior performance, mainly when data are implicitly non-IID, and PLMs scale larger.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

24 of 45 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  7. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  8. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
  9. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  10. 2015LSTM: A Search Space Odysseycited by 316
  11. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  12. 2019Text Classification Algorithms: A Surveycited by 163
12 more
  1. 2018Deep Learning for Sentiment Analysis : A Surveycited by 139
  2. 2022KronA: Parameter Efficient Tuning with Kronecker Adaptercited by 133
  3. 2017Adversarial Multi-task Learning for Text Classificationcited by 91
  4. 2023Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systemscited by 45
  5. 2014A Unified Perspective on Multi-Domain and Multi-Task Learningcited by 33
  6. 2022Bayesian Learning for Neural Networks: an algorithmic surveycited by 23
  7. 2024Personalized LoRA for Human-Centered Text Understandingcited by 17
  8. 2018Improving Review Representations with User Attention and Product Attention for Sentiment Classificationcited by 8
  9. 2019Rethinking Attribute Representation and Injection for Sentiment Classificationcited by 8
  10. 2020Improving Document-Level Sentiment Analysis with User and Product Contextcited by 7
  11. 2021Learning to Share by Masking the Non-shared for Multi-domain Sentiment Classificationcited by 3
  12. 2022A Curriculum Learning Approach for Multi-domain Text Classification Using Keyword weight Rankingcited by 2

Built on by

Not cited yet.