mlsanityData to 7 Oct 2026

Generalizing Trust: Weak-to-Strong Trustworthiness in Language Models

arXiv 2501.00418 ↗First posted 31 Dec 2024cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

The rapid proliferation of generative AI, especially large language models, has led to their integration into a variety of applications. A key phenomenon known as weak-to-strong generalization - where a strong model trained on a weak model's outputs surpasses the weak model in task performance - has gained significant attention. Yet, whether critical trustworthiness properties such as robustness, fairness, and privacy can generalize similarly remains an open question. In this work, we study this question by examining if a stronger model can inherit trustworthiness properties when fine-tuned on a weaker model's outputs, a process we term weak-to-strong trustworthiness generalization. To address this, we introduce two foundational training strategies: 1) Weak Trustworthiness Finetuning (Weak TFT), which leverages trustworthiness regularization during the fine-tuning of the weak model, and 2) Weak and Weak-to-Strong Trustworthiness Finetuning (Weak+WTS TFT), which extends regularization to both weak and strong models. Our experimental evaluation on real-world datasets reveals that while some trustworthiness properties, such as fairness, adversarial, and OOD robustness, show significant improvement in transfer when both models were regularized, others like privacy do not exhibit signs of weak-to-strong trustworthiness. As the first study to explore trustworthiness generalization via weak-to-strong generalization, our work provides valuable insights into the potential and limitations of weak-to-strong generalization.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 31 Dec 2024.

Builds on

37 of 57 references.

References · 37

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  11. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  12. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
25 more
  1. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  2. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  3. 2015Generating Sentences from a Continuous Spacecited by 607
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  5. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  6. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  7. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  8. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  9. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  10. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  11. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
  12. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  13. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  14. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  15. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  16. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  17. 2018Certified Adversarial Robustness with Additive Noisecited by 178
  18. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  19. 2019Fair Generative Modeling via Weak Supervisioncited by 114
  20. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  21. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  22. 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecturecited by 63
  23. 2017Learning the Structure of Generative Models without Labeled Datacited by 50
  24. 2023Students Parrot Their Teachers: Membership Inference on Model Distillationcited by 31
  25. 2023Gaussian Membership Inference Privacycited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Weak-to-Strong Generalization via Bregman Bias-Variance Decompositioncited by 3
  2. 2025Weak-to-Strong Generalization with Failure Trajectories: A Tree-based Approach to Elicit Optimal Policy in Strong Modelscited by 3
  3. 2025The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibrationcited by 2
  4. 2025Limitations of refinement methods for weak to strong generalizationcited by 1
  5. 2025From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0cited by 0
  6. 2026On the Blessing of Pre-training in Weak-to-Strong Generalizationcited by 0