mlsanityData to 14 Oct 2026

LLaVaOLMoBitnet1B: Ternary LLM goes Multimodal!

arXiv 2408.13402 ↗First posted 23 Aug 2024cs.LGcited by 3

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MM-LLMs) have seen significant advancements in the last year, demonstrating impressive performance across tasks. However, to truly democratize AI, models must exhibit strong capabilities and be able to run efficiently on small compute footprints accessible by most. Part of this quest, we introduce LLaVaOLMoBitnet1B - the first Ternary Multimodal LLM capable of accepting Image(s)+Text inputs to produce coherent textual responses. The model is fully open-sourced along with training scripts to encourage further research in this space. This accompanying technical report highlights the training process, evaluation details, challenges associated with ternary models and future opportunities. Link to the model: https://huggingface.co/IntelLabs/LlavaOLMoBitnet1B

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 11 references that are in the map. First posted on arXiv 23 Aug 2024.

Builds on

10 of 11 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Models22 Feb

Other references · 9

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  6. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  7. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  8. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
  9. 2024LLaVA-Gemma: Accelerating Multimodal Foundation Models with a Compact Language Modelcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantizationcited by 1
  2. 2025Binary Neural Networks for Large Language Model: A Surveycited by 1
  3. 2026Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inferencecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024MM-LLMs: Recent Advances in MultiModal Large Language Models
  2. 2026Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
  3. 2024A Review of Multi-Modal Large Language and Vision Models
  4. 2024LLMs Meet Multimodal Generation and Editing: A Survey
  5. 2024Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges
  6. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
  7. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey
  8. 2026TokenSwap: Benchmarking and Reducing the Modality Gap in Multimodal LLMs