mlsanityData to 14 Oct 2026

(N,K)-Puzzle: A Cost-Efficient Testbed for Benchmarking Reinforcement Learning Algorithms in Generative Language Model

arXiv 2403.07191 ↗First posted 11 Mar 2024cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in reinforcement learning (RL) algorithms aim to enhance the performance of language models at scale. Yet, there is a noticeable absence of a cost-effective and standardized testbed tailored to evaluating and comparing these algorithms. To bridge this gap, we present a generalized version of the 24-Puzzle: the (N,K)-Puzzle, which challenges language models to reach a target value K with N integers. We evaluate the effectiveness of established RL algorithms such as Proximal Policy Optimization (PPO), alongside novel approaches like Identity Policy Optimization (IPO) and Direct Policy Optimization (DPO).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 18 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

14 of 18 references.

References · 14

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2020Learning to summarize from human feedbackcited by 2,588
  12. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
2 more
  1. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  2. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CE-RM: A Pointwise Generative Reward Model Optimized via Two-Stage Rollout and Unified Criteria
  2. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Models
  3. 2025RAPID: An Efficient Reinforcement Learning Algorithm for Small Language Models
  4. 2025Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data
  5. 2024Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement
  6. 2026Learning with a Single Rollout via Pass@k Policy Optimization
  7. 2025Studying the Korean Word-Chain Game with RLVR: Mitigating Reward Conflicts via Curriculum Learning
  8. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization