mlsanityData to 14 Oct 2026

Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model

arXiv 2505.24476 ↗First posted 30 May 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Periodic or quasi-periodic phenomena reveal intrinsic characteristics in various natural processes, such as weather patterns, movement behaviors, traffic flows, and biological signals. Given that these phenomena span multiple modalities, the capabilities of Multimodal Large Language Models (MLLMs) offer promising potential to effectively capture and understand their complex nature. However, current MLLMs struggle with periodic tasks due to limitations in: 1) lack of temporal modelling and 2) conflict between short and long periods. This paper introduces Period-LLM, a multimodal large language model designed to enhance the performance of periodic tasks across various modalities, and constructs a benchmark of various difficulty for evaluating the cross-modal periodic capabilities of large models. Specially, We adopt an "Easy to Hard Generalization" paradigm, starting with relatively simple text-based tasks and progressing to more complex visual and multimodal tasks, ensuring that the model gradually builds robust periodic reasoning capabilities. Additionally, we propose a "Resisting Logical Oblivion" optimization strategy to maintain periodic reasoning abilities during semantic alignment. Extensive experiments demonstrate the superiority of the proposed Period-LLM over existing MLLMs in periodic tasks. The code is available at https://github.com/keke-nice/Period-LLM.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 52 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

34 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023OneLLM: One Framework to Align All Modalities with Language6 Dec

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  9. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  10. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  11. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  12. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
21 more
  1. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  2. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  3. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  4. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 334
  5. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 234
  6. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  7. 2023Teaching CLIP to Count to Tencited by 198
  8. 2020Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurementcited by 134
  9. 2020MetNet: A Neural Weather Model for Precipitation Forecastingcited by 131
  10. 2019RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representationcited by 105
  11. 2018Class-Agnostic Countingcited by 85
  12. 2020Counting Out Time: Class Agnostic Video Repetition Counting in the Wildcited by 68
  13. 2024Video Anomaly Detection and Explanation via Large Language Modelscited by 47
  14. 2022TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action Countingcited by 44
  15. 2022SimPer: Simple Self-Supervised Learning of Periodic Targetscited by 44
  16. 2023rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurementcited by 37
  17. 2018DeepDownscale: a Deep Learning Strategy for High-Resolution Weather Forecastcited by 28
  18. 2023Image Enhancement for Remote Photoplethysmography in a Low-Light Environmentcited by 23
  19. 2024GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computingcited by 22
  20. 2021The First Vision For Vitals (V4V) Challenge for Non-Contact Video-Based Physiological Estimationcited by 19
  21. 2024Advancing Generalizable Remote Physiological Measurement through the Integration of Explicit and Implicit Prior Knowledgecited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MM-LLMs: Recent Advances in MultiModal Large Language Models
  2. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Models
  3. 2025Reasoning is Periodicity? Improving Large Language Models Through Effective Periodicity Modeling
  4. 2023A Comprehensive Overview of Large Language Models
  5. 2026DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
  6. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
  7. 2024Towards Time Series Reasoning with LLMs
  8. 2025Aligning Multimodal LLM with Human Preference: A Survey