mlsanityData to 7 Oct 2026

ABC Align: Large Language Model Alignment for Safety & Accuracy

arXiv 2408.00307 ↗First posted 1 Aug 2024cs.LG cs.AI cs.CLcited by 3

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Alignment of Large Language Models (LLMs) remains an unsolved problem. Human preferences are highly distributed and can be captured at multiple levels of abstraction, from the individual to diverse populations. Organisational preferences, represented by standards and principles, are defined to mitigate reputational risk or meet legislative obligations. In this paper, we present ABC Align, a novel alignment methodology for LLMs that enables integration of the standards and preferences of a large media organisation into the LLM itself. We combine a set of data and methods that build on recent breakthroughs in synthetic data generation, preference optimisation, and post-training model quantisation. Our unified approach mitigates bias and improves accuracy, while preserving reasoning capability, as measured against standard benchmarks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 34 of its 53 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

34 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024ORPO: Monolithic Preference Optimization without Reference Model12 Mar
  2. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning4 Dec
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  4. 2022Training language models to follow instructions with human feedback4 Mar
  5. 2021A General Language Assistant as a Laboratory for Alignment1 Dec

Other references · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  12. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
17 more
  1. 2024The Platonic Representation Hypothesiscited by 362
  2. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  3. 2023Fine-tuning Language Models for Factualitycited by 219
  4. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 187
  5. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  6. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  7. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
  8. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  9. 2019Synthetic Data for Deep Learningcited by 113
  10. 2023LLM-Rec: Personalized Recommendation via Prompting Large Language Modelscited by 77
  11. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  12. 2023Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Taskscited by 59
  13. 2021A Few More Examples May Be Worth Billions of Parameterscited by 19
  14. 2023Hyperparameter Optimization for Large Language Model Instruction-Tuningcited by 17
  15. 2024Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?cited by 11
  16. 2024Reinforcement learning for question answering in programming domain using public community scoring as a human feedbackcited by 6
  17. 2024Guiding the Way: A Comprehensive Examination of AI Guidelines in Global Mediacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Position: We Need Responsible, Application-Driven (RAD) AI Researchcited by 2
  2. 2025ENIGMA: The Geometry of Reasoning and Alignment in Large-Language Modelscited by 1
  3. 2026ATLAS: Constitution-Conditioned Latent Geometry and Redistribution Across Language Models and Neural Perturbation Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  2. 2025Data Selection for LLM Alignment Using Fine-Grained Preferences
  3. 2024MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time
  4. 2023Aligning Large Language Models with Human: A Survey
  5. 2025Leveraging Robust Optimization for LLM Alignment under Distribution Shifts
  6. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models
  7. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications
  8. 2023On Diversified Preferences of Large Language Model Alignment