mlsanityData to 7 Oct 2026

Control Barrier Function for Aligning Large Language Models

arXiv 2511.03121 ↗First posted 5 Nov 2025cs.CL cs.AI cs.SYcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

This paper proposes a control-based framework for aligning large language models (LLMs) by leveraging a control barrier function (CBF) to ensure user-desirable text generation. The presented framework applies the CBF safety filter to the predicted token generated from the baseline LLM, to intervene in the generated text. The safety filter includes two significant advantages: this safety filter is an add-on type, allowing it to be used for alignment purposes without fine-tuning the baseline LLM, and if there is an evaluation model regarding the desired alignment, it can be directly applied to the filter design. The overall text-generation system is implemented with open-source language models, aiming to generate positive text.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 39 of its 63 references that are in the map. First posted on arXiv 5 Nov 2025.

Builds on

39 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct

Other references · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  9. 2023LIMA: Less Is More for Alignmentcited by 1,080
  10. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  11. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  12. 2024Large Language Models: A Surveycited by 506
26 more
  1. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  2. 2020Autoregressive Entity Retrievalcited by 333
  3. 2019Control Barrier Functions: Theory and Applicationscited by 332
  4. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  5. 2023Large Language Model Alignment: A Surveycited by 241
  6. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  7. 2022Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methodscited by 148
  8. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
  9. 2023Controlled Decoding from Language Modelscited by 133
  10. 2022TimeLMs: Diachronic Language Models from Twittercited by 129
  11. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  12. 2023Aligning Large Language Models through Synthetic Feedbackcited by 75
  13. 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
  14. 2020Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Functioncited by 42
  15. 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
  16. 2019Adaptive Safety with Control Barrier Functionscited by 25
  17. 2023Taming AI Bots: Controllability of Neural States in Large Language Modelscited by 22
  18. 2024Is In-Context Learning Sufficient for Instruction Following in LLMs?cited by 19
  19. 2024Unlearning Traces the Influential Training Data of Language Modelscited by 18
  20. 2022Control Barrier Functions and Input-to-State Safety with Application to Automated Vehiclescited by 11
  21. 2019Control Barrier Functions for Mechanical Systems: Theory and Application to Robotic Graspingcited by 8
  22. 2024Safety-Critical Control for Autonomous Systems: Control Barrier Functions via Reduced-Order Modelscited by 6
  23. 2022Control Barrier Functions for Stochastic Systems and Safety-critical Control Designscited by 4
  24. 2024Language Model Sentence Completion with a Parser-Driven Rhetorical Control Methodcited by 3
  25. 2022Distributed Coverage Hole Prevention for Visual Environmental Monitoring with Quadcopters via Nonsmooth Control Barrier Functionscited by 2
  26. 2024Probabilistically safe controllers based on control barrier functions and scenario model predictive controlcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024CBF-LLM: Safe Control for LLM Alignment
  2. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching
  3. 2025Where to Start Alignment? Diffusion Large Language Model May Demand a Distinct Position
  4. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations
  5. 2023On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused?
  6. 2026DNAlign: Dynamic Null-Space Safe Alignment for LLMs
  7. 2024LongSafety: Enhance Safety for Long-Context LLMs
  8. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements