mlsanityData to 14 Oct 2026

GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learning

arXiv 2511.15256 ↗First posted 19 Nov 2025cs.LG cs.CVcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

The Group Relative Policy Optimization (GRPO), a reinforcement learning method used to fine-tune large language models (LLMs), has proved its effectiveness in practical applications such as DeepSeek-R1. It raises a question whether GRPO can be generalized to representation learning models. In this paper, we propose Group Relative Policy Optimization for Representation Model (GRPO-RM), and investigate the performance of GRPO-like policy in post-training representation models. Specifically, our method establishes a predefined output set to functionally replace token sequence sampling in LLMs, thereby generating an output group, which is essential for the probability-driven optimization of GRPO. In addition, a specialized reward function is designed to accommodate the properties of representation models. Extensive experiments are conducted on various real-world datasets to validate the effectiveness of our proposed method.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 50 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

42 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  4. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  8. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  9. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  10. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  11. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  12. 2020End-to-End Object Detection with Transformerscited by 7,414
26 more
  1. 2017Mask R-CNNcited by 7,277
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  7. 2015Trust Region Policy Optimizationcited by 2,580
  8. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  9. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  10. 2018Path Aggregation Network for Instance Segmentationcited by 798
  11. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  12. 2020Meta Pseudo Labelscited by 374
  13. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  14. 2019Semantic Correlation Promoted Shape-Variant Context for Segmentationcited by 61
  15. 2022Positive-incentive Noisecited by 39
  16. 2023Variational Positive-incentive Noise: How Noise Benefits Modelscited by 38
  17. 2024Enhance Vision-Language Alignment with Noisecited by 34
  18. 2024Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noisecited by 25
  19. 2025Learn Beneficial Noise as Graph Augmentationcited by 15
  20. 2024Comparative Analysis of ImageNet Pre-Trained Deep Learning Models and DINOv2 in Medical Imaging Classificationcited by 14
  21. 2025NFIG: Multi-Scale Autoregressive Image Generation via Frequency Orderingcited by 14
  22. 2025Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learningcited by 14
  23. 2024DINOv2 based Self Supervised Learning For Few Shot Medical Image Segmentationcited by 11
  24. 2025Explore How to Inject Beneficial Noise in MLLMscited by 10
  25. 2025Rectified Noise: A Generative Model Using Positive-incentive Noisecited by 7
  26. 2025Laytrol: Preserving Pretrained Knowledge in Layout Control for Multimodal Diffusion Transformerscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learningcited by 10
  2. 2026GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
  2. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
  3. 2026GRPO Training Dynamics for Small Language Models
  4. 2025GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control
  5. 2025Composing Policy Gradients and Prompt Optimization for Language Model Programs
  6. 2025Sharpness-Guided Group Relative Policy Optimization via Probability Shaping
  7. 2026MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Models
  8. 2025Group Sequence Policy Optimization