# The Robot's Inner Critic: Self-Refinement of Social Behaviors through VLM-based Replanning

arXiv: https://arxiv.org/abs/2603.20164 · HTML: https://mlsanity.com/paper/2603.20164/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Builds on (most cited first, 11 of 11)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots](https://mlsanity.com/paper/1810.12541/) (2018)
- [RePLan: Robotic Replanning with Perception and Language Models](https://mlsanity.com/paper/2401.04157/) (2024)
- [From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control](https://mlsanity.com/paper/2405.04798/) (2024)
- [ReplanVLM: Replanning Robotic Tasks with Visual Language Models](https://mlsanity.com/paper/2407.21762/) (2024)
- [Generative Expressive Robot Behaviors using Large Language Models](https://mlsanity.com/paper/2401.14673/) (2024)
- [Multi-Agent LLM Actor-Critic Framework for Social Robot Navigation](https://mlsanity.com/paper/2503.09758/) (2025)
- [Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment](https://mlsanity.com/paper/2412.04835/) (2024)
- [EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning](https://mlsanity.com/paper/2410.23234/) (2024)
- [Towards Embedding Dynamic Personas in Interactive Robots: Masquerading Animated Social Kinematics (MASK)](https://mlsanity.com/paper/2403.10041/) (2024)

## Built on by (most cited first, 1 of 1)

- [Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops](https://mlsanity.com/paper/2607.07663/) (2026)

## Abstract (arXiv)

Conventional robot social behavior generation has been limited in flexibility and autonomy, relying on predefined motions or human feedback. This study proposes CRISP (Critique-and-Replan for Interactive Social Presence), an autonomous framework where a robot critiques and replans its own actions by leveraging a Vision-Language Model (VLM) as a `human-like social critic.' CRISP integrates (1) extraction of movable joints and constraints by analyzing the robot's description file (e.g., MJCF), (2) generation of step-by-step behavior plans based on situational context, (3) generation of low-level joint control code by referencing visual information (joint range-of-motion visualizations), (4) VLM-based evaluation of social appropriateness and naturalness, including pinpointing erroneous steps, and (5) iterative refinement of behaviors through reward-based search. This approach is not tied to a specific robot API; it can generate subtly different, human-like motions on various platforms using only the robot's structure file. In a user study involving five different robot types and 20 scenarios, including mobile manipulators and humanoids, our proposed method achieved significantly higher preference and situational appropriateness ratings compared to previous methods. This research presents a general framework that minimizes human intervention while expanding the robot's autonomous interaction capabilities and cross-platform applicability. Detailed result videos and supplementary information regarding this work are available at: https://limjiyu99.github.io/inner-critic/