mlsanityData to 14 Oct 2026

Entity Embeddings : Perspectives Towards an Omni-Modality Era for Large Language Models

arXiv 2310.18390 ↗First posted 27 Oct 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are evolving to integrate multiple modalities, such as text, image, and audio into a unified linguistic space. We envision a future direction based on this framework where conceptual entities defined in sequences of text can also be imagined as modalities. Such a formulation has the potential to overcome the cognitive and computational limitations of current models. Several illustrative examples of such potential implicit modalities are given. Along with vast promises of the hypothesized structure, expected challenges are discussed as well.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

22 of 39 references.

References · 22

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  8. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  9. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  10. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  11. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  12. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
10 more
  1. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  2. 2023GPT-NER: Named Entity Recognition via Large Language Modelscited by 160
  3. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  4. 2023X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languagescited by 128
  5. 2022Language Models are General-Purpose Interfacescited by 97
  6. 2023AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Modelcited by 93
  7. 2023Transferable Models for Bioacoustics with Human Language Supervisioncited by 21
  8. 2022End-to-End Table Question Answering via Retrieval-Augmented Generationcited by 14
  9. 2023Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Modelscited by 12
  10. 2023Learning to Imagine: Visually-Augmented Natural Language Generationcited by 7

Built on by

Not cited yet.