mlsanityData to 14 Oct 2026

Online-PVLM: Advancing Personalized VLMs with Online Concept Learning

arXiv 2511.20056 ↗First posted 25 Nov 2025cs.CLcited by 5

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Personalized Visual Language Models (VLMs) are gaining increasing attention for their formidable ability in user-specific concepts aligned interactions (e.g., identifying a user's bike). Existing methods typically require the learning of separate embeddings for each new concept, which fails to support real-time adaptation during testing. This limitation becomes particularly pronounced in large-scale scenarios, where efficient retrieval of concept embeddings is not achievable. To alleviate this gap, we propose Online-PVLM, a framework for online concept learning by leveraging hyperbolic representations. Our approach makes a train-free paradigm for concept embeddings generation at test time, making the use of personalized VLMs both scalable and efficient. In addition, we develop OP-Eval, a comprehensive and large-scale benchmark comprising 1,292 concepts and over 30K high-quality instances with diverse question types, designed to rigorously assess online concept learning in realistic scenarios. Extensive experiments demonstrate the state-of-the-art performance of our proposed framework. Our source code and dataset will be made available.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 26 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

11 of 26 references.

References · 11

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  5. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  6. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  7. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  8. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  9. 2024Yo'LLaVA: Your Personalized Language and Vision Assistantcited by 62
  10. 2019Combining MixMatch and Active Learning for Better Accuracy with Fewer Labelscited by 17
  11. 2024MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Modelscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026M2A: Multimodal Memory Agent with Dual-Layer Hybrid Memory for Long-Term Personalized Interactionscited by 25
  2. 2026Personal Visual Context Learning in Large Multimodal Modelscited by 3
  3. 2026Group Preference Collapse in Personalized Multimodal Large Language Modelscited by 0
  4. 2026Parametric Multimodal User Memory: Storing What Captions Cannot Carrycited by 0
  5. 2026Anti-Persona: Disrupting Unauthorized Identity Binding and Recognition in Personalized Vision--Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MyVLM: Personalizing VLMs for User-Specific Queries
  2. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks
  3. 2025Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs
  4. 2025Data Metabolism: An Efficient Data Design Schema For Vision Language Model
  5. 2025Small-Large Collaboration: Training-efficient Concept Personalization for Large VLM using a Meta Personalized Small VLM
  6. 2024MC-LLaVA: Multi-Concept Personalized Vision-Language Model
  7. 2024Understanding Visual Concepts Across Models
  8. 2025Visual Large Language Models for Generalized and Specialized Applications