mlsanityData to 14 Oct 2026

Language-enhanced RNR-Map: Querying Renderable Neural Radiance Field maps with natural language

arXiv 2308.08854 ↗First posted 17 Aug 2023cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

We present Le-RNR-Map, a Language-enhanced Renderable Neural Radiance map for Visual Navigation with natural language query prompts. The recently proposed RNR-Map employs a grid structure comprising latent codes positioned at each pixel. These latent codes, which are derived from image observation, enable: i) image rendering given a camera pose, since they are converted to Neural Radiance Field; ii) image navigation and localization with astonishing accuracy. On top of this, we enhance RNR-Map with CLIP-based embedding latent codes, allowing natural language search without additional label data. We evaluate the effectiveness of this map in single and multi-object searches. We also investigate its compatibility with a Large Language Model as an "affordance query resolver". Code and videos are available at https://intelligolabs.github.io/Le-RNR-Map/

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 17 Aug 2023.

Builds on

27 of 30 references.

References · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  5. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  6. 2022Language-driven Semantic Segmentationcited by 717
  7. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  8. 2023LERF: Language Embedded Radiance Fieldscited by 543
  9. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  10. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  11. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  12. 2022Visual Language Maps for Robot Navigationcited by 365
15 more
  1. 2020Learning to Explore using Active Neural SLAMcited by 316
  2. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  3. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  4. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  5. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 160
  6. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  7. 2019Learning Exploration Policies for Navigationcited by 116
  8. 2019Embodied Question Answering in Photorealistic Environments with Point Cloud Perceptioncited by 103
  9. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  10. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 55
  11. 2023Renderable Neural Radiance Map for Visual Navigationcited by 51
  12. 2022Semantic-Conditional Diffusion Networks for Image Captioningcited by 38
  13. 2023AutoNeRF: Training Implicit Scene Representations with Autonomous Agentscited by 10
  14. 2023OO-dMVMT: A Deep Multi-view Multi-task Classification Framework for Real-time 3D Hand Gesture Classification and Segmentationcited by 3
  15. 2023Unsupervised Active Visual Search with Monte Carlo planning under Uncertain Detectionscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 5.

5 papers.

  1. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 65
  2. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 22
  3. 2024Neural Fields in Robotics: A Surveycited by 21
  4. 2024Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialoguescited by 8
  5. 2024I2EDL: Interactive Instruction Error Detection and Localizationcited by 1