mlsanityData to 7 Oct 2026

Contrast with Major Classifier Vectors for Federated Medical Relation Extraction with Heterogeneous Label Distribution

arXiv 2301.05376 ↗First posted 13 Jan 2023cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated medical relation extraction enables multiple clients to train a deep network collaboratively without sharing their raw medical data. In order to handle the heterogeneous label distribution across clients, most of the existing works only involve enforcing regularization between local and global models during optimization. In this paper, we fully utilize the models of all clients and propose a novel concept of textitmajor classifier vectors, where a group of class vectors is obtained in an ensemble rather than the weighted average method on the server. The major classifier vectors are then distributed to all clients and the local training of each client is Contrasted with Major Classifier vectors (FedCMC), so the local model is not prone to overfitting to the local label distribution. FedCMC requires only a small amount of additional transfer of classifier parameters without any leakage of raw data, extracted representations, and label distributions. Our extensive experiments show that FedCMC outperforms the other state-of-the-art FL algorithms on three medical relation extraction datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 54 references that are in the map. First posted on arXiv 13 Jan 2023.

Builds on

32 of 54 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2019Advances and Open Problems in Federated Learningcited by 3,021
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  9. 2019Publicly Available Clinical BERT Embeddingscited by 995
  10. 2021Model-Contrastive Federated Learningcited by 762
  11. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  12. 2020The Future of Digital Health with Federated Learningcited by 625
20 more
  1. 2020Long-tail learning via logit adjustmentcited by 605
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  3. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  4. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  5. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  6. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 284
  7. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  8. 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 271
  9. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  10. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  11. 2018Graph Convolution over Pruned Dependency Trees Improves Relation Extractioncited by 146
  12. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  13. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  14. 2021Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognitioncited by 95
  15. 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
  16. 2020Federated pretraining and fine tuning of BERT using clinical notes from multiple siloscited by 18
  17. 2020Modeling Topical Relevance for Multi-Turn Dialogue Generationcited by 15
  18. 2019A Silver Standard Corpus of Human Phenotype-Gene Relationscited by 13
  19. 2019Leveraging Dependency Forest for Neural Medical Relation Extractioncited by 10
  20. 2020Distantly Supervised Relation Extraction in Federated Settingscited by 3

Built on by

Not cited yet.