# SplatR : Experience Goal Visual Rearrangement with 3D Gaussian Splatting and Dense Feature Matching

arXiv: https://arxiv.org/abs/2411.14322 · HTML: https://mlsanity.com/paper/2411.14322/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 4 of its 52 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 4 of 4)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [UniTeam: Open Vocabulary Mobile Manipulation Challenge](https://mlsanity.com/paper/2312.08611/) (2023)
- [Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge](https://mlsanity.com/paper/2407.06939/) (2024)
- [Language-Conditioned Semantic Search-Based Policy for Robotic Manipulation Tasks](https://mlsanity.com/paper/2312.05925/) (2023)

## Built on by (most cited first, 2 of 2)

- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [Grounding Language Models with Semantic Digital Twins for Robotic Planning](https://mlsanity.com/paper/2506.16493/) (2025)

## Abstract (arXiv)

Experience Goal Visual Rearrangement task stands as a foundational challenge within Embodied AI, requiring an agent to construct a robust world model that accurately captures the goal state. The agent uses this world model to restore a shuffled scene to its original configuration, making an accurate representation of the world essential for successfully completing the task. In this work, we present a novel framework that leverages on 3D Gaussian Splatting as a 3D scene representation for experience goal visual rearrangement task. Recent advances in volumetric scene representation like 3D Gaussian Splatting, offer fast rendering of high quality and photo-realistic novel views. Our approach enables the agent to have consistent views of the current and the goal setting of the rearrangement task, which enables the agent to directly compare the goal state and the shuffled state of the world in image space. To compare these views, we propose to use a dense feature matching method with visual features extracted from a foundation model, leveraging its advantages of a more universal feature representation, which facilitates robustness, and generalization. We validate our approach on the AI2-THOR rearrangement challenge benchmark and demonstrate improvements over the current state of the art methods