mlsanityData to 14 Oct 2026

PRISM: Preference Refinement via Implicit Scene Modeling for 3D Vision-Language Preference-Based Reinforcement Learning

arXiv 2503.10177 ↗First posted 13 Mar 2025cs.CL cs.ROcited by 1

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

We propose PRISM, a novel framework designed to overcome the limitations of 2D-based Preference-Based Reinforcement Learning (PBRL) by unifying 3D point cloud modeling and future-aware preference refinement. At its core, PRISM adopts a 3D Point Cloud-Language Model (3D-PC-LLM) to mitigate occlusion and viewpoint biases, ensuring more stable and spatially consistent preference signals. Additionally, PRISM leverages Chain-of-Thought (CoT) reasoning to incorporate long-horizon considerations, thereby preventing the short-sighted feedback often seen in static preference comparisons. In contrast to conventional PBRL techniques, this integration of 3D perception and future-oriented reasoning leads to significant gains in preference agreement rates, faster policy convergence, and robust generalization across unseen robotic environments. Our empirical results, spanning tasks such as robotic manipulation and autonomous navigation, highlight PRISM's potential for real-world applications where precise spatial understanding and reliable long-term decision-making are critical. By bridging 3D geometric awareness with CoT-driven preference modeling, PRISM establishes a comprehensive foundation for scalable, human-aligned reinforcement learning.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 11 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

9 of 11 references.

References · 9

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2020Point Transformercited by 1,470
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2023Mastering Diverse Domains through World Modelscited by 1,253
  8. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  9. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 156

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Modelcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
  2. 2025Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learning
  3. 2026Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation ↗
  4. 2025The Impact of VR and 2D Interfaces on Human Feedback in Preference-Based Robot Learning
  5. 2025ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representations
  6. 2026PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learning
  7. 2026Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment
  8. 2024Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment