mlsanityData to 7 Oct 2026

Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models

arXiv 2505.17826 ↗First posted 23 May 2025cs.LG cs.CL cs.DCcited by 19

Not on the map. Why?

Abstract (arXiv)

Trinity-RFT is a general-purpose, unified and easy-to-use framework designed for reinforcement fine-tuning (RFT) of large language models. It is built with a modular and decoupled design, consisting of (1) an RFT-core that unifies and generalizes synchronous/asynchronous, on-policy/off-policy, and online/offline modes of RFT; (2) seamless integration for agent-environment interaction with high efficiency and robustness; and (3) systematic data pipelines optimized for RFT. Trinity-RFT can be easily adapted for diverse application scenarios, and serves as a unified platform for development and research of advanced reinforcement learning paradigms at both macroscopic and microscopic levels. This technical report outlines the vision, features, design and implementations of Trinity-RFT, accompanied by extensive examples, applications and experiments that demonstrate its functionalities and user-friendliness.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

29 of 50 references.

References · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  12. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
17 more
  1. 2018Experience Replay for Continual Learningcited by 1,074
  2. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  3. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  4. 2025Learning to Reason under Off-Policy Guidancecited by 266
  5. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  6. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
  7. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 120
  8. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 117
  9. 2024DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agentscited by 68
  10. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 65
  11. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 56
  12. 2023Data-Juicer: A One-Stop Data Processing System for Large Language Modelscited by 52
  13. 2024Offline Regularised Reinforcement Learning for Large Language Models Alignmentcited by 48
  14. 2024Contrastive Policy Gradient: Aligning LLMs on sequence-level scores in a supervised-friendly fashioncited by 13
  15. 2025Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Datacited by 13
  16. 2025MindGYM: What Matters in Question Synthesis for Thinking-Centric Fine-Tuning?cited by 6
  17. 2025An Expectation-Maximization Perspective on Reinforcement Learning for LLM Reasoningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 13.

19 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  2. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learningcited by 94
  3. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  4. 2026TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agentscited by 38
  5. 2026On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agentscited by 21
  6. 2025Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friendscited by 13
  7. 2025BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuningcited by 12
  8. 2026R3L: Reflect-then-Retry Reinforcement Learning with Language-Guided Exploration, Pivotal Credit, and Positive Amplificationcited by 12
  9. 2026On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Modelscited by 8
  10. 2025Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logscited by 3
  11. 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learningcited by 1
  12. 2026DASH-OPD: Discrepancy-Aware Switching with Hysteresis for On-Policy Distillationcited by 1
7 more
  1. 2026Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidancecited by 0
  2. 2026Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learningcited by 0
  3. 2026GEOALIGN: Geometric Rollout Curation for Robust LLM Reinforcement Learningcited by 0
  4. 2026Learning Stateful Predictive Knowledge From Experiencecited by 0
  5. 2026UOPD: Uncertainty-Aware Intervention for On-Policy Distillation of Multi-Turn Agentscited by 0
  6. 2026Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamicscited by 0
  7. 2026Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policynesscited by 0