mlsanityData to 7 Oct 2026

MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environments

arXiv 2203.07060 ↗First posted 14 Mar 2022cs.CV cs.ROcited by 40

On the map: World models → World modeling from LiDAR generation

Abstract (arXiv)

This work addresses a gap in semantic scene completion (SSC) data by creating a novel outdoor data set with accurate and complete dynamic scenes. Our data set is formed from randomly sampled views of the world at each time step, which supervises generalizability to complete scenes without occlusions or traces. We create SSC baselines from state-of-the-art open source networks and construct a benchmark real-time dense local semantic mapping algorithm, MotionSC, by leveraging recent 3D deep learning architectures to enhance SSC with temporal information. Our network shows that the proposed data set can quantify and supervise accurate scene completion in the presence of dynamic objects, which can lead to the development of improved dynamic mapping algorithms. All software is available at https://github.com/UMich-CURLY/3DMapping.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Placed in World modeling from LiDAR generation, one of 11 directions in World models. Builds on 46 of its 62 references that are in the map. First posted on arXiv 14 Mar 2022.

Builds on

46 of 62 references.

Closest ancestors · 2

Same direction, newest first

  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space7 Jun
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation2 Dec

Other references · 44

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  8. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  9. 2016End to End Learning for Self-Driving Carscited by 1,000
  10. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  11. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  12. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
32 more
  1. 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
  2. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  3. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  4. 2017SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloudcited by 231
  5. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  6. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  7. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  8. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  9. 2020SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instancescited by 123
  10. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  11. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  12. 2017Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigationcited by 90
  13. 2017Deep Projective 3D Semantic Segmentationcited by 81
  14. 2020Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Datacited by 74
  15. 20213D Semantic Scene Completion: a Surveycited by 74
  16. 2018Learning Navigation Behaviors End-to-End with AutoRLcited by 70
  17. 2019Efficient Semantic Scene Completion Network with Spatial Group Convolutioncited by 66
  18. 2021Semantic Scene Completion via Integrating Instances and Scene in-the-Loopcited by 54
  19. 2018Two Stream 3D Semantic Scene Completioncited by 48
  20. 2019Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mappingcited by 34
  21. 2021ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point Cloud Map Buildingcited by 29
  22. 2018View-volume Network for Semantic Scene Completion from a Single Depth Imagecited by 26
  23. 2019SynthCity: A large scale synthetic point cloudcited by 26
  24. 2021Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mappingcited by 25
  25. 2016A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time Applicationcited by 17
  26. 2017DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networkscited by 17
  27. 2018Recurrent-OctoMap: Learning State-based Map Refinement for Long-Term Semantic Mapping with 3D-Lidar Datacited by 16
  28. 2021NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mappingcited by 14
  29. 2019SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequencescited by 11
  30. 2017Dense RGB-D semantic mapping with Pixel-Voxel neural networkcited by 4
  31. 2021Self-Supervised Simultaneous Multi-Step Prediction of Road Dynamics and Cost Mapcited by 4
  32. 2021Dynamic Semantic Occupancy Mapping using 3D Scene Flow and Closed-Form Bayesian Inferencecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 16; 2025: 12; 2026: 5.

40 papers.

Sort:

By area:

  1. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  2. 20253D and 4D World Modeling: A Surveycited by 64
  3. 2025A Survey of World Models for Autonomous Drivingcited by 60
  4. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  5. 2023Pyramid Diffusion for Fine 3D Large Scene Generationcited by 44
  6. 2024Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillationcited by 43
  7. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 37
  8. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Datacited by 33
  9. 2024OccMamba: Semantic Occupancy Prediction with State Space Modelscited by 29
  10. 2024CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillationcited by 27
  11. 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 27
  12. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
28 more
  1. 20253D Scene Generation: A Surveycited by 24
  2. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  3. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  4. 2023ConvBKI: Real-Time Probabilistic Semantic Mapping Network with Quantifiable Uncertaintycited by 10
  5. 2024LiRaFusion: Deep Adaptive LiDAR-Radar Fusion for 3D Object Detectioncited by 10
  6. 2025La La LiDAR: Large-Scale Layout Generation from LiDAR Datacited by 10
  7. 2024Label-efficient Semantic Scene Completion with Scribble Annotationscited by 9
  8. 2024PanoSSC: Exploring Monocular Panoptic 3D Scene Reconstruction for Autonomous Drivingcited by 9
  9. 2024MemFusionMap: Working Memory Fusion for Online Vectorized HD Map Constructioncited by 9
  10. 2025Controllable 3D Outdoor Scene Generation via Scene Graphscited by 9
  11. 2022Convolutional Bayesian Kernel Inference for 3D Semantic Mappingcited by 8
  12. 2024BeautyMap: Binary-Encoded Adaptable Ground Matrix for Dynamic Points Removal in Global Mapscited by 6
  13. 2024ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learningcited by 6
  14. 2024UFO: Uncertainty-aware LiDAR-image Fusion for Off-road Semantic Terrain Map Estimationcited by 5
  15. 2025Towards Generating Realistic 3D Semantic Training Data for Autonomous Drivingcited by 5
  16. 2024Deep Learning for 3D Point Cloud Enhancement: A Surveycited by 4
  17. 2026Extend3D: Town-Scale 3D Generationcited by 4
  18. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  19. 2022SoLo T-DIRL: Socially-Aware Dynamic Local Planner based on Trajectory-Ranked Deep Inverse Reinforcement Learningcited by 2
  20. 2024Uncertainty-aware Semantic Mapping in Off-road Environments with Dempster-Shafer Theory of Evidencecited by 2
  21. 2024LatentBKI: Open-Dictionary Continuous Mapping in Visual-Language Latent Spaces with Quantifiable Uncertaintycited by 2
  22. 2025Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotatorcited by 1
  23. 2025PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenescited by 1
  24. 2026An Instance-Centric Panoptic Occupancy Prediction Benchmark for Autonomous Drivingcited by 1
  25. 2026SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Datacited by 1
  26. 2024Lift, Splat, Map: Lifting Foundation Masks for Label-Free Semantic Scene Completioncited by 0
  27. 2026OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Spacecited by 0
  28. 2026Generative Semantic Scene Completioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fields
  2. 2025HD2-SSC: High-Dimension High-Density Semantic Scene Completion for Autonomous Driving
  3. 2026Memory Over Maps: 3D Object Localization Without Reconstruction
  4. 2026Vision-Language-Motion Maps: An Open-Vocabulary, Uncertainty-Aware, Queryable Motion Attribute for 3D Scene Maps ↗
  5. 2025DSM: Constructing a Diverse Semantic Map for 3D Visual Grounding
  6. 2026Open-Set 3D Scene Graphs for Field Robotics: An Outdoor Case Study ↗
  7. 2026Think While You Map: Asynchronous Vision-Language Agents for Incremental 3D Scene Graphs
  8. 2026A Scene Language Model for Open-Vocabulary Scene Mapping