mlsanityData to 7 Oct 2026

Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learning

arXiv 2512.06533 ↗First posted 6 Dec 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Decoding-based regression, which reformulates regression as a sequence generation task, has emerged as a promising paradigm of applying large language models for numerical prediction. However, its progress is hindered by the misalignment between discrete token-level objectives (e.g., cross-entropy) and continuous numerical values. Existing approaches relying on token-level constraints often fail to capture the global magnitude of the target value, limiting their precision and generalization. In this paper, we propose to unlock the potential of decoding-based regression via Reinforcement Learning (RL). We formulate the generation process as a Markov Decision Process, utilizing sequence-level rewards to enforce global numerical coherence. Extensive experiments on tabular regression and code metric regression demonstrate that our method (specifically with ReMax and GRPO) consistently outperforms both state-of-the-art token-level baselines and traditional regression heads, showing the superiority of introducing sequence-level signals. Our analysis further reveals that RL significantly enhances sampling efficiency and predictive precision, establishing decoding-based regression as a robust and accurate paradigm for general-purpose numerical prediction.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 77 of its 107 references that are in the map. First posted on arXiv 6 Dec 2025.

Builds on

77 of 107 references.

References · 77

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  8. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  9. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  10. 2017Deep reinforcement learning from human preferencescited by 3,846
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  12. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
38 more
  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  2. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  3. 2021Measuring Coding Challenge Competence With APPScited by 879
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  5. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  6. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  7. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  8. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  9. 2018Conditional Neural Processescited by 421
  10. 2021Transformers Can Do Bayesian Inferencecited by 338
  11. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  12. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
  13. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 184
  14. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  15. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  16. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  17. 2024Generative Reward Modelscited by 148
  18. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  19. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 134
  20. 2024ODIN: Disentangled Reward Mitigates Hacking in RLHFcited by 133
  21. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 126
  22. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 115
  23. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  24. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 103
  25. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 84
  26. 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 82
  27. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 70
  28. 2024Self-Generated Critiques Boost Reward Modeling for Language Modelscited by 69
  29. 2025Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Modelscited by 67
  30. 2018Improving Regression Performance with Distributional Lossescited by 66
  31. 2025LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligencecited by 65
  32. 2021Linear algebra with transformerscited by 62
  33. 2024From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examplescited by 55
  34. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  35. 2025The Path Not Taken: RLVR Provably Learns Off the Principalscited by 52
  36. 2017Deep Forestcited by 46
  37. 2024TALENT: A Tabular Analytics and Learning Toolboxcited by 46
  38. 2025Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelscited by 46

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Reinforcement Learning over Predictive Distributions for LLM Regressioncited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework
  2. 2024Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework
  3. 2026REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge
  4. 2026Adaptive Decoding via Test-Time Policy Learning for Self-Improving Generation
  5. 2025Diversity or Precision? A Deep Dive into Next Token Prediction
  6. 2023Controlled Decoding from Language Models
  7. 2026The Optimal Token Baseline: Variance Reduction for Long-Horizon LLM-RL
  8. 2026Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling