mlsanityData to 14 Oct 2026

NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding

arXiv 2507.20110 ↗First posted 27 Jul 2025cs.CV cs.AI cs.LGcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recent breakthroughs in Visual Language Models (VLMs) and Multimodal Large Language Models (MLLMs) have significantly advanced 3D scene perception towards language-driven cognition. However, existing 3D language models struggle with sparse, large-scale point clouds due to slow feature extraction and limited representation accuracy. To address these challenges, we propose NeuroVoxel-LM, a novel framework that integrates Neural Radiance Fields (NeRF) with dynamic resolution voxelization and lightweight meta-embedding. Specifically, we introduce a Dynamic Resolution Multiscale Voxelization (DR-MSV) technique that adaptively adjusts voxel granularity based on geometric and structural complexity, reducing computational cost while preserving reconstruction fidelity. In addition, we propose the Token-level Adaptive Pooling for Lightweight Meta-Embedding (TAP-LME) mechanism, which enhances semantic representation through attention-based weighting and residual fusion. Experimental results demonstrate that DR-MSV significantly improves point cloud feature extraction efficiency and accuracy, while TAP-LME outperforms conventional max-pooling in capturing fine-grained semantics from NeRF weights.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 63 of its 72 references that are in the map. First posted on arXiv 27 Jul 2025.

Builds on

63 of 72 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generation5 Dec
  2. 2023CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes12 Apr

Other references · 61

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  10. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
38 more
  1. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  3. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  4. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  5. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  6. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  7. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  8. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  9. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  10. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  11. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  12. 2023LERF: Language Embedded Radiance Fieldscited by 543
  13. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  14. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  15. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  16. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  17. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  18. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  19. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  20. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
  21. 2023SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesiscited by 247
  22. 2017Submanifold Sparse Convolutional Networkscited by 223
  23. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 218
  24. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 202
  25. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 194
  26. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  27. 2023ImageBind-LLM: Multi-modality Instruction Tuningcited by 142
  28. 2022NeRF-Art: Text-Driven Neural Radiance Fields Stylizationcited by 126
  29. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
  30. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  31. 2022Is Attention All That NeRF Needs?cited by 113
  32. 2020Predicting Neural Network Accuracy from Weightscited by 110
  33. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 107
  34. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  35. 2023End-to-End 3D Dense Captioning with Vote2Cap-DETRcited by 78
  36. 2022X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioningcited by 70
  37. 2023Deep Learning on Implicit Neural Representations of Shapescited by 63
  38. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layoutcited by 57

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understanding
  2. 2026SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation
  3. 2025LangScene-X: Reconstruct Generalizable 3D Language-Embedded Scenes with TriMap Video Diffusion
  4. 2025OpenVox: Real-time Instance-level Open-vocabulary Probabilistic Voxel Representation
  5. 2025VoxRep: Enhancing 3D Spatial Understanding in 2D Vision-Language Models via Voxel Representation
  6. 20243DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding
  7. 2026GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models
  8. 20243DBench: A Scalable 3D Benchmark and Instruction-Tuning Dataset