mlsanityData to 7 Oct 2026

Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values

arXiv 2210.07652 ↗First posted 14 Oct 2022cs.CL cs.AIcited by 13

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Many NLP classification tasks, such as sexism/racism detection or toxicity detection, are based on human values. Yet, human values can vary under diverse cultural conditions. Therefore, we introduce a framework for value-aligned classification that performs prediction based on explicitly written human values in the command. Along with the task, we propose a practical approach that distills value-aligned knowledge from large-scale language models (LLMs) to construct value-aligned classifiers in two steps. First, we generate value-aligned training data from LLMs by prompt-based few-shot learning. Next, we fine-tune smaller classification models with the generated data for the task. Empirical results show that our VA-Models surpass multiple baselines by at least 15.56% on the F1-score, including few-shot learning with OPT-175B and existing text augmentation methods. We suggest that using classifiers with explicit human value input improves both inclusivity & explainability in AI.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 29 of its 49 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

29 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Aligning AI With Shared Human Values5 Aug

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
16 more
  1. 2021Ethical and social risks of harm from Language Modelscited by 957
  2. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  3. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  4. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  5. 2020Artificial Intelligence, Values and Alignmentcited by 443
  6. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  7. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  8. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  9. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  10. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  11. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  12. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 100
  13. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 67
  14. 2020Language Models as Few-Shot Learner for Task-Oriented Dialogue Systemscited by 41
  15. 2019Multi-label Categorization of Accounts of Sexism using a Neural Frameworkcited by 19
  16. 2021Ethics Sheets for AI Taskscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 3; 2025: 2; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  2. 2023TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Modelscited by 56
  3. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  4. 2023SemEval-2023 Task 10: Explainable Detection of Online Sexismcited by 48
  5. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  6. 2024ComPO: Community Preferences for Language Model Personalizationcited by 19
  7. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  8. 2024Evaluating Moral Beliefs across LLMs through a Pluralistic Frameworkcited by 11
  9. 2025Evaluation and Facilitation of Online Discussions in the LLM Era: A Surveycited by 8
  10. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  11. 2024SPICA: Retrieving Scenarios for Pluralistic In-Context Alignmentcited by 6
  12. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
1 more
  1. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models
  2. 2025Internal Value Alignment in Large Language Models through Controlled Value Vector Activation
  3. 2024Beyond Labels: Aligning Large Language Models with Human-like Reasoning
  4. 2025C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models
  5. 2026VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models
  6. 2025Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation
  7. 2025DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values
  8. 2025Diverse Human Value Alignment for Large Language Models via Ethical Reasoning