mlsanityData to 14 Oct 2026

DualReward: A Dynamic Reinforcement Learning Framework for Cloze Tests Distractor Generation

arXiv 2507.11875 ↗First posted 16 Jul 2025cs.CLcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

This paper introduces DualReward, a novel reinforcement learning framework for automatic distractor generation in cloze tests. Unlike conventional approaches that rely primarily on supervised learning or static generative models, our method employs a dual reward structure with adaptive scaling that differentiates between human-created gold standard distractors and model-generated candidates. The framework dynamically adjusts reward signal intensity based on model performance and confidence. We evaluate our approach on both passage-level (CLOTH-F) and sentence-level (MCQ) cloze test datasets, demonstrating consistent improvements over state-of-the-art baselines. Experimental results show that our adaptive reward scaling mechanism provides modest but consistent benefits on homogeneous datasets (CLOTH-F) and more substantial improvements (3.48-3.86% in P@1) on diverse, cross-domain data (MCQ), suggesting its particular effectiveness for handling varied question types and domains. Our work offers a flexible framework that effectively balances learning from reliable human examples while exploring novel, high-quality distractors for automated test generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 24 references that are in the map. First posted on arXiv 16 Jul 2025.

Builds on

15 of 24 references.

References · 15

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  4. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  5. 2017Inverse Reward Designcited by 249
  6. 2018Meta-Gradient Reinforcement Learningcited by 159
  7. 2017Sentence Simplification with Deep Reinforcement Learningcited by 109
  8. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 102
  9. 2018Reinforcement Learning with Perturbed Rewardscited by 79
  10. 2024Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluationcited by 25
  11. 2020Knowledge-Driven Distractor Generation for Cloze-style Multiple Choice Questionscited by 24
  12. 2020A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategiescited by 22
3 more
  1. 2024Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decodingcited by 11
  2. 2024Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integrationcited by 11
  3. 2024Adaptive Reward Design for Reinforcement Learningcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIF
  2. 2026RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time
  3. 2025TTRL: Test-Time Reinforcement Learning
  4. 2024CARMO: Dynamic Criteria Generation for Context-Aware Reward Modelling
  5. 2024Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation
  6. 2026Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design
  7. 2025OneReward: Unified Mask-Guided Image Generation via Multi-Task Human Preference Learning
  8. 2025Better Language Model-Based Judging Reward Modeling through Scaling Comprehension Boundaries