mlsanityData to 7 Oct 2026

SELU: Self-Learning Embodied MLLMs in Unknown Environments

arXiv 2410.03303 ↗First posted 4 Oct 2024cs.LG cs.CVcited by 6

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recently, multimodal large language models (MLLMs) have demonstrated strong visual understanding and decision-making capabilities, enabling the exploration of autonomously improving MLLMs in unknown environments. However, external feedback like human or environmental feedback is not always available. To address this challenge, existing methods primarily focus on enhancing the decision-making capabilities of MLLMs through voting and scoring mechanisms, while little effort has been paid to improving the environmental comprehension of MLLMs in unknown environments. To fully unleash the self-learning potential of MLLMs, we propose a novel actor-critic self-learning paradigm, dubbed SELU, inspired by the actor-critic paradigm in reinforcement learning. The critic employs self-asking and hindsight relabeling to extract knowledge from interaction trajectories collected by the actor, thereby augmenting its environmental comprehension. Simultaneously, the actor is improved by the self-feedback provided by the critic, enhancing its decision-making. We evaluate our method in the AI2-THOR and VirtualHome environments, and SELU achieves critic improvements of approximately 28% and 30%, and actor improvements of about 20% and 24% via self-learning.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 61 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

49 of 61 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation17 Jun
  2. 2024CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automation19 Feb
  3. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback6 Feb
  4. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation24 Dec

Other references · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
33 more
  1. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  3. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  4. 2022Large Language Models Can Self-Improvecited by 656
  5. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  6. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  7. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
  8. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  9. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  10. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  11. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  12. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  13. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  14. 2024LLM Critics Help Catch LLM Bugscited by 138
  15. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
  16. 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Modelcited by 119
  17. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 98
  18. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  19. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answeringcited by 93
  20. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
  21. 2022PEER: A Collaborative Language Modelcited by 86
  22. 2024Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Taskscited by 83
  23. 2022Language Models Can Teach Themselves to Program Bettercited by 75
  24. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 75
  25. 2024Unified Hallucination Detection for Multimodal Large Language Modelscited by 75
  26. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  27. 2022Read before Generate! Faithful Long Form Question Answering with Machine Readingcited by 47
  28. 2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agentscited by 41
  29. 2024Optimizing RLHF Training for Large Language Models with Stage Fusioncited by 41
  30. 2022Unsupervised Learning of Group Invariant and Equivariant Representationscited by 38
  31. 2023Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Modelscited by 31
  32. 2023BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Trainingcited by 24
  33. 2023PACE: Improving Prompt with Actor-Critic Editing for Large Language Modelcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  2. 2025World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planningcited by 26
  3. 2025The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvementcited by 19
  4. 2025Self-Improvement in Multimodal Large Language Models: A Surveycited by 6
  5. 2026Echoes Over Time: Unlocking Length Generalization in Video-to-Audio Generation Modelscited by 1
  6. 2026Exponential Approximation Rates and Parameter Efficiency of Learnable Bernstein Activationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SERL: Self-Examining Reinforcement Learning on Open-Domain
  2. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  3. 2026Experiential Reinforcement Learning
  4. 2026SEIF: Self-Evolving Reinforcement Learning for Instruction Following
  5. 2026EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
  6. 2025A Survey on Agentic Multimodal Large Language Models
  7. 2025Vision-Language Model Dialog Games for Self-Improvement
  8. 2025From Emergence to Control: Probing and Modulating Self-Reflection in Language Models