# IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes arXiv: https://arxiv.org/abs/2112.05298 ยท HTML: https://mlsanity.com/paper/2112.05298/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 64 of its 98 references that are in the map. - First posted on arXiv 10 Dec 2021. ## Builds on (most cited first, 30 of 64) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016) - [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017) - [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016) - [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017) - [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020) - [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018) - [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018) - [Scene Graph Generation by Iterative Message Passing](https://mlsanity.com/paper/1701.02426/) (2017) - [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016) - [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019) - [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015) - [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020) - [RoboTHOR: An Open Simulation-to-Real Embodied AI Platform](https://mlsanity.com/paper/2004.06799/) (2020) - [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021) - [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021) - [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019) - [Rearrangement: A Challenge for Embodied AI](https://mlsanity.com/paper/2011.01975/) (2020) - [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016) - [Scene Graph Generation from Objects, Phrases and Region Captions](https://mlsanity.com/paper/1707.09700/) (2017) - [ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging](https://mlsanity.com/paper/1904.06830/) (2019) - [Grounded Human-Object Interaction Hotspots from Video](https://mlsanity.com/paper/1812.04558/) (2018) - [AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection](https://mlsanity.com/paper/1709.07326/) (2017) - [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018) - [CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction](https://mlsanity.com/paper/2012.00924/) (2020) - [Deep Learning for Detecting Robotic Grasps](https://mlsanity.com/paper/1301.3592/) (2013) - [Learning Exploration Policies for Navigation](https://mlsanity.com/paper/1903.01959/) (2019) - [EGO-TOPO: Environment Affordances from Egocentric Video](https://mlsanity.com/paper/2001.04583/) (2020) - [Self-supervised 6D Object Pose Estimation for Robot Manipulation](https://mlsanity.com/paper/1909.10159/) (2019) ## Built on by (most cited first, 9 of 9) - [LEGO-Net: Learning Regular Rearrangements of Objects in Rooms](https://mlsanity.com/paper/2301.09629/) (2023) - [Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces](https://mlsanity.com/paper/2503.19199/) (2025) - [LLM-enhanced Scene Graph Learning for Household Rearrangement](https://mlsanity.com/paper/2408.12093/) (2024) - [Learning Reusable Manipulation Strategies](https://mlsanity.com/paper/2311.03293/) (2023) - [MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planning](https://mlsanity.com/paper/2512.16909/) (2025) - [FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning](https://mlsanity.com/paper/2604.03696/) (2026) - [BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment](https://mlsanity.com/paper/2207.08192/) (2022) - [Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces](https://mlsanity.com/paper/2605.15753/) (2026) - [PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning](https://mlsanity.com/paper/2606.08655/) (2026) ## Abstract (arXiv) Building embodied intelligent agents that can interact with 3D indoor environments has received increasing research attention in recent years. While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e.g., a switch on the wall turns on or off the light, a remote control operates the TV). Humans often spend little or no effort to infer these relationships, even when entering a new room, by using our strong prior knowledge (e.g., we know that buttons control electrical devices) or using only a few exploratory interactions in cases of uncertainty (e.g., multiple switches and lights in the same room). In this paper, we take the first step in building AI system learning inter-object functional relationships in 3D indoor environments with key technical contributions of modeling prior knowledge by training over large-scale scenes and designing interactive policies for effectively exploring the training scenes and quickly adapting to novel test scenes. We create a new benchmark based on the AI2Thor and PartNet datasets and perform extensive experiments that prove the effectiveness of our proposed method. Results show that our model successfully learns priors and fast-interactive-adaptation strategies for exploring inter-object functional relationships in complex 3D scenes. Several ablation studies further validate the usefulness of each proposed module.