mlsanityData to 7 Oct 2026

InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training

arXiv 2510.15859 ↗First posted 17 Oct 2025cs.CL cs.AIcited by 11

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Reinforcement learning (RL) has powered many recent breakthroughs in large language models (LLMs), especially for tasks where rewards can be computed automatically, such as code generation. However, it is less effective in open-ended medical dialogue, where feedback is ambiguous, context-dependent, and difficult to simply summarize into a single scalar signal-often requiring heavily supervised reward models and creating risks of reward hacking. Thus, we introduce ORBIT, an open-ended rubric-based incremental training framework tailored for critical medical dialogues. ORBIT integrates medical dialogue construction with dynamically generated case-conditioned rubrics that serve as adaptive guides for incremental RL. Unlike approaches that rely on external medical knowledge bases or handcrafted rules, ORBIT uses rubric-guided evaluation and can be implemented with general-purpose instruction-following LLMs, avoiding task-specific judge fine-tuning. With only 2k training samples, ORBIT raises Qwen3-4B-Instruct's HealthBench-Hard score from 7.0 to 27.5, achieving state-of-the-art performance among similarly sized open-source models while maintaining strong consultation quality as rubric coverage broadens.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 50 of its 72 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

50 of 72 references.

References · 50

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  8. 2024Qwen2.5 Technical Reportcited by 4,713
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  10. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  11. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  12. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
38 more
  1. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  2. 2025Group Sequence Policy Optimizationcited by 788
  3. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  4. 2024Large Language Models: A Surveycited by 506
  5. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  6. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
  7. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
  8. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  9. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 300
  10. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
  11. 2023Reward Design with Language Modelscited by 257
  12. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  13. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
  14. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 210
  15. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 187
  16. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 171
  17. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  18. 2024InFoBench: Evaluating Instruction Following Ability in Large Language Modelscited by 128
  19. 2023Towards Accurate Differential Diagnosis with Large Language Modelscited by 115
  20. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 115
  21. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 105
  22. 2022Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Supportcited by 68
  23. 2025Baichuan-M2: Scaling Medical Capability with Large Verifier Systemcited by 55
  24. 2020MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generationcited by 45
  25. 2022A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasetscited by 45
  26. 2025m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Modelscited by 42
  27. 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
  28. 2025Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RLcited by 34
  29. 2024MEDCO: Medical Education Copilots Based on A Multi-Agent Frameworkcited by 33
  30. 2025QA-LIGN: Aligning LLMs through Constitutionally Decomposed QAcited by 28
  31. 2025Real-World Doctor Agent with Proactive Consultation through Multi-Agent Reinforcement Learningcited by 26
  32. 2025ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoningcited by 26
  33. 2025Language Models that Think, Chat Bettercited by 23
  34. 2021ReMeDi: Resources for Multi-domain, Multi-service, Medical Dialoguescited by 20
  35. 2023LLM-driven Multimodal Target Volume Contouring in Radiation Oncologycited by 20
  36. 2025Ask Patients with Patience: Enabling LLMs for Human-Centric Medical Dialogue with Grounded Reasoningcited by 19
  37. 2025Compute as Teacher: Turning Inference Compute Into Reference-Free Supervisioncited by 17
  38. 2025ACE-RL: Adaptive Constraint-Enhanced Reward for Long-form Generation Reinforcement Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 55
  2. 2026Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Trainingcited by 45
  3. 2026Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learningcited by 9
  4. 2026Health-SCORE: Towards Scalable Rubrics for Improving Health-LLMscited by 7
  5. 2026Grad2Reward: From Sparse Judgment to Dense Rewards for Improving Open-Ended LLM Reasoningcited by 5
  6. 2026From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscapecited by 4
  7. 2026Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewardscited by 3
  8. 2026ClinAlign: Scaling Healthcare Alignment from Clinician Preferencecited by 2
  9. 2026ProMedical: Hierarchical Fine-Grained Criteria Modeling for Medical LLM Alignment via Explicit Injectioncited by 2
  10. 2026ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agentscited by 2
  11. 2026ConRub-Med: Reinforcement Learning with Consensus Rubrics for Open-Ended Medical Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards
  2. 2023Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue
  3. 2026MediX-R1: Open Ended Medical Reinforcement Learning
  4. 2025Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations
  5. 2025Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning
  6. 2026Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO
  7. 2025Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branching
  8. 2024Exploring LLM-based Data Annotation Strategies for Medical Dialogue Preference Alignment