mlsanityData to 7 Oct 2026

CircuitLens: Reasoning Circuits as Data Selection Signals for Reinforcement Learning with Verifiable Rewards

arXiv 2609.07183 ↗First posted 7 Sep 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) is sensitive to which problems a model trains on, yet existing selection criteria--difficulty filtering, hand-curation, reward-trajectory scoring--assess data value as an intrinsic property of problems, independent of the model that will learn from them. We introduce Circuit Reasoning Score (CRS), a selection signal derived from 46 reasoning-sensitive attention heads identified via contrastive ablation, computed in a single forward pass on the frozen base model without reward labels or rollouts. CRS runs against the intuitive hypothesis that stronger reasoning-circuit engagement produces better training data: on Qwen2.5-Math-7B, the lowest-engagement decile improves over random selection on three medium-difficulty benchmarks (GSM8K +2.0 pp, OlympiadBench +1.6 pp, Minerva +2.9 pp), while the highest-engagement decile gains less and is indistinguishable from the middle decile. The advantage has boundary conditions: on a domain-curated pool no selection method separates from the others; at 1.5B scale the useful direction differs; and the lowest-reward training condition produces the strongest downstream generalization. Within the Qwen2.5-Math settings tested, RLVR data selection appears regime-dependent rather than reducible to a static ranking of problem quality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

31 of 38 references.

References · 31

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  8. 2022In-context Learning and Induction Headscited by 930
  9. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  10. 2025LIMO: Less is More for Reasoningcited by 488
  11. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  12. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
19 more
  1. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
  2. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 276
  3. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
  4. 2024How to use and interpret activation patchingcited by 191
  5. 2024Free Process Rewards without Process Labelscited by 162
  6. 2025LIMR: Less is More for RL Scalingcited by 136
  7. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 134
  8. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
  9. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 103
  10. 2025Language Models Use Trigonometry to Do Additioncited by 68
  11. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  12. 2025Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Explorationcited by 40
  13. 2024Data Selection via Optimal Control for Language Modelscited by 38
  14. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 35
  15. 2024Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Modelscited by 32
  16. 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inferencecited by 28
  17. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decompositioncited by 23
  18. 2025Causal Head Gating: A Framework for Interpreting Roles of Attention Heads in Transformerscited by 16
  19. 2025CircuitSeer: Mining High-Quality Data by Probing Mathematical Reasoning Circuits in LLMscited by 3

Built on by

Not cited yet.