mlsanityData to 7 Oct 2026

BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning

arXiv 2603.04124 ↗First posted 4 Mar 2026cs.AI cond-mat.mtrl-sci cs.CLcited by 4

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Can reinforcement learning with hard, verifiable rewards teach a compact language model to reason about physics, or does it primarily learn to pattern-match toward correct answers? We study this question by training a 1.5B-parameter reasoning model on beam statics, a classic engineering problem, using parameter-efficient RLVR with binary correctness rewards from symbolic solvers, without teacher-generated reasoning traces. The best BeamPERL checkpoint achieves a 66.7% improvement in Pass@1 over the base model. However, the learned competence is anisotropic: the model generalizes compositionally (more loads) but fails under topological shifts (moved supports) that require the same equilibrium equations. Intermediate checkpoints yield the strongest reasoning, while continued optimization degrades robustness while maintaining reward. These findings reveal a key limitation of outcome-level alignment: reinforcement learning with exact physics rewards induces procedural solution templates rather than internalization of governing equations. The precision of the reward signal - even when analytically exact - does not by itself guarantee transferable physical reasoning. Our results suggest that verifiable rewards may need to be paired with structured reasoning scaffolding to move beyond template matching toward robust scientific reasoning.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 63 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

34 of 63 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Models10 Sep
  2. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  3. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  5. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  6. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't20 Mar
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  8. 2025Training Language Models to Reason Efficiently6 Feb
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  10. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  8. 2024OpenAI o1 System Cardcited by 1,220
  9. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  10. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  11. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  12. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
12 more
  1. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Modelscited by 78
  2. 2024X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Designcited by 41
  3. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  4. 2025Tina: Tiny Reasoning Models via LoRAcited by 23
  5. 2025Deep Research: A Systematic Surveycited by 23
  6. 2025Sparks: Multi-Agent Artificial Intelligence Model Discovers Protein Design Principlescited by 16
  7. 2025Integrating Large Language Models for Automated Structural Analysiscited by 12
  8. 2025In-situ graph reasoning and knowledge expansion using Graph-PReFLexORcited by 11
  9. 2026Reasoning Models Generate Societies of Thoughtcited by 11
  10. 2026GraphAgents: Knowledge Graph-Guided Agentic AI for Cross-Domain Materials Designcited by 9
  11. 2024Parameter Efficient Reinforcement Learning from Human Feedbackcited by 7
  12. 2023AI for Science: An Emerging Agendacited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Autonomous Agents Coordinating Distributed Discovery Through Emergent Artifact Exchangecited by 11
  2. 2026Graph-Native Reinforcement Learning Enables Traceable Scientific Hypothesis Generation through Conceptual Recombinationcited by 3
  3. 2026Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Modelcited by 0
  4. 2026OraclePhys: A Systematic Framework for LLM Fine-Tuning on Structural Mechanicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards
  2. 2026QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learning
  3. 2026Decoupled Physical Modeling and Execution for Physics Reasoning
  4. 2026Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models
  5. 2026SARL: Label-Free Reinforcement Learning by Rewarding Reasoning Topology
  6. 2026SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions
  7. 2026SCOPE-RL: Optimizing Reasoning Paths Before and After Success
  8. 2025Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning