RedRFT: A Light-Weight Benchmark for Reinforcement Fine-Tuning-Based Red Teaming
On the map: LLM safety and security → Prompt injection and agent security
Abstract (arXiv)
Red teaming has proven to be an effective method for identifying and mitigating vulnerabilities in Large Language Models (LLMs). Reinforcement Fine-Tuning (RFT) has emerged as a promising strategy among existing red teaming techniques. However, a lack of a unified benchmark hinders current RFT-based red teaming methods. Implementation details, especially in Proximal Policy Optimization (PPO)-based RFT, significantly affect outcome stability and reproducibility. To address this issue, we introduce RedRFT, a lightweight benchmark designed to simplify and standardize the implementation and evaluation of RFT-based red teaming. RedRFT combines the design strengths of both single-file CleanRL and highly modularized Tianshou, offering high-quality single-file red teaming implementations and modular PPO core components, such as the General Advantage Estimator. It supports a variety of token and sentence diversity metrics, featuring modularized intrinsic reward computation that facilitates plug-and-play experimentation. To clarify their influence on RFT performance, we conducted an extensive ablation study on key components, including Low-Rank Adaptation (LoRA), Kullback-Leibler (KL) divergence, and Lagrange Multiplier. We hope this work contributes to 1) gaining a comprehensive understanding of the implementation nuances of RFT-based red teaming algorithms, and 2) enabling rapid prototyping of innovative features for RFT-based red teaming. Code for the benchmark can be accessed at https://github.com/x-zheng16/RedRFT.git.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 41 of its 45 references that are in the map. First posted on arXiv 4 Jun 2025.
Builds on
41 of 45 references.
Closest ancestors · 7
Same direction, newest first
- 2024DiveR-CT: Diversity-enhanced Red Teaming Large Language Model Assistants with Relaxing Constraints29 May
- 2024Curiosity-driven Red-teaming for Large Language Models29 Feb
- 2023Attack Prompt Generation for Red Teaming and Defending Large Language Models19 Oct
- 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratch15 Jun
- 2023Query-Efficient Black-Box Red Teaming via Bayesian Optimization27 May
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned23 Aug
- 2022Red Teaming Language Models with Language Models7 Feb
Other references · 34
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024GPT-4o System Cardcited by 4,588
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2018Exploration by Random Network Distillationcited by 993
- 2023Large Language Models as Optimizerscited by 864
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
22 more
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
- 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
- 2021Unsolved Problems in ML Safetycited by 292
- 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
- 2016Variational Intrinsic Controlcited by 267
- 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
- 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
- 2018Provably Efficient Maximum Entropy Explorationcited by 242
- 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
- 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 162
- 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
- 2021APS: Active Pretraining with Successor Featurescited by 112
- 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
- 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 68
- 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
- 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
- 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
- 2024Constrained Intrinsic Motivation for Reinforcement Learningcited by 6
- 2025CALM: Curiosity-Driven Auditing for Large Language Modelscited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025RedTopic: Toward Topic-Diverse Red Teaming of Large Language Models
- 2024Learning diverse attacks on large language models for robust red-teaming and safety tuning
- 2025Automatic LLM Red Teaming
- 2026CART: Closed-Loop Adaptive Red Teaming for Large Language Models
- 2025Be a Multitude to Itself: A Prompt Evolution Framework for Red Teaming
- 2026Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models
- 2026ARES: Adaptive Red-Teaming and End-to-End Repair of Policy-Reward System
- 2024Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learning