mlsanityData to 7 Oct 2026

SoMeLVLM: A Large Vision Language Model for Social Media Processing

arXiv 2402.13022 ↗First posted 20 Feb 2024cs.CL cs.MMcited by 11

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

The growth of social media, characterized by its multimodal nature, has led to the emergence of diverse phenomena and challenges, which calls for an effective approach to uniformly solve automated tasks. The powerful Large Vision Language Models make it possible to handle a variety of tasks simultaneously, but even with carefully designed prompting methods, the general domain models often fall short in aligning with the unique speaking style and context of social media tasks. In this paper, we introduce a Large Vision Language Model for Social Media Processing (SoMeLVLM), which is a cognitive framework equipped with five key capabilities including knowledge & comprehension, application, analysis, evaluation, and creation. SoMeLVLM is designed to understand and generate realistic social media behavior. We have developed a 654k multimodal social media instruction-tuning dataset to support our cognitive framework and fine-tune our model. Our experiments demonstrate that SoMeLVLM achieves state-of-the-art performance in multiple social media tasks. Further analysis shows its significant advantages over baselines in terms of cognitive abilities.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 75 references that are in the map. First posted on arXiv 20 Feb 2024.

Builds on

35 of 75 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  5. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  6. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  7. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  8. 2017Fake News Detection on Social Media: A Data Mining Perspectivecited by 334
  9. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
  10. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 216
  11. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
  12. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
23 more
  1. 2019Automatically Neutralizing Subjective Bias in Textcited by 105
  2. 2019Exploring Hate Speech Detection in Multimodal Publicationscited by 103
  3. 2020Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representationscited by 97
  4. 2020Stance Detection on Social Media: State of the Art and Trendscited by 93
  5. 2020We Can Detect Your Bias: Predicting the Political Ideology of News Articlescited by 92
  6. 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 78
  7. 2023DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuningcited by 64
  8. 2017A Large Self-Annotated Corpus for Sarcasmcited by 63
  9. 2016Stance and Sentiment in Tweetscited by 62
  10. 2023Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmarkcited by 62
  11. 2018Modeling Empathy and Distress in Reaction to News Storiescited by 52
  12. 2019Humor Detection: A Transformer Gets the Last Laughcited by 46
  13. 2020Detecting Hate Speech in Multi-modal Memescited by 38
  14. 2021Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlinescited by 32
  15. 2019Automatically Identifying Complaints in Social Mediacited by 25
  16. 2020Quantifying Intimacy in Languagecited by 25
  17. 2021Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexicacited by 23
  18. 2020SemEval-2020 Task 7: Assessing Humor in Edited News Headlinescited by 15
  19. 2021MOVER: Mask, Over-generate and Rank for Hyperbole Generationcited by 13
  20. 2022Understanding and Detecting Hateful Content using Contrastive Learningcited by 13
  21. 2022MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learningcited by 13
  22. 2023ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarkscited by 12
  23. 2020Facilitating the Communication of Politeness through Fine-Grained Paraphrasingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 4.

11 papers.

  1. 2025SocioVerse: A World Model for Social Simulation Powered by LLM Agents and A Pool of 10 Million Real-World Userscited by 50
  2. 2024ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agentscited by 35
  3. 2025Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understandingcited by 7
  4. 2024INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurancecited by 6
  5. 2025SNS-Bench-VL: Benchmarking Multimodal Large Language Models in Social Networking Servicescited by 2
  6. 2024Overview of the CAIL 2023 Argument Mining Trackcited by 0
  7. 2025MERIT: Modular Framework for Multimodal Misinformation Detection with Web-Grounded Reasoningcited by 0
  8. 2026Fine-grained Verbal Attack Detection via a Hierarchical Divide-and-Conquer Frameworkcited by 0
  9. 2026SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Contextcited by 0
  10. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challengescited by 0
  11. 2026Towards High-Level Semantic Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
  2. 2025LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
  3. 2024MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms
  4. 2024Uncovering Bias in Large Vision-Language Models with Counterfactuals
  5. 2026GraphVLM: Benchmarking Vision Language Models for Multimodal Graph Learning
  6. 2023Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Model
  7. 2026Video-Only ToM: Enhancing Theory of Mind in Multimodal Large Language Models
  8. 2024HumanVLM: Foundation for Human-Scene Vision-Language Model