mlsanityData to 14 Oct 2026

Predicting Future Instance Segmentation by Forecasting Convolutional Features

arXiv 1803.11496 ↗First posted 30 Mar 2018cs.CVcited by 37

Not on the map. Why?

Abstract (arXiv)

Anticipating future events is an important prerequisite towards intelligent behavior. Video forecasting has been studied as a proxy task towards this goal. Recent work has shown that to predict semantic segmentation of future frames, forecasting at the semantic level is more effective than forecasting RGB frames and then segmenting these. In this paper we consider the more challenging problem of future instance segmentation, which additionally segments out individual objects. To deal with a varying number of output labels per image, we develop a predictive model in the space of fixed-sized convolutional features of the Mask R-CNN instance segmentation model. We apply the "detection head'" of Mask R-CNN on the predicted features to produce the instance segmentation of future frames. Experiments show that this approach significantly improves over strong baselines based on optical flow and repurposed instance segmentation architectures.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 14; 2023: 3; 2024: 4; 2025: 4.

37 papers.

Sort:

By area:

  1. 2021Anticipative Video Transformercited by 200
  2. 2021Predicting Video with VQVAEcited by 61
  3. 2025Back to the Features: DINO as a Foundation for Video World Modelscited by 58
  4. 2024DINO-Foresight: Looking into the Future with DINOcited by 45
  5. 2021Learning to Anticipate Egocentric Actions by Imaginationcited by 40
  6. 2022Real-time Object Detection for Streaming Perceptioncited by 30
  7. 2022Forecasting from LiDAR via Future Object Detectioncited by 28
  8. 2021Revisiting Hierarchical Approach for Persistent Long-Term Video Predictioncited by 23
  9. 2021Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videoscited by 23
  10. 2022Generative Adversarial Network for Future Hand Segmentation from Egocentric Videocited by 16
  11. 2022MSPred: Video Prediction at Multiple Spatio-Temporal Scales with Hierarchical Recurrent Networkscited by 13
  12. 2021Dense Out-of-Distribution Detection by Robust Learning on Synthetic Negative Datacited by 10
25 more
  1. 2022Long-term Video Frame Interpolation via Feature Propagationcited by 8
  2. 2025Advancing Semantic Future Prediction through Multimodal Visual Sequence Transformerscited by 8
  3. 2021Panoptic Segmentation Forecastingcited by 7
  4. 2021Warp-Refine Propagation: Semi-Supervised Auto-labeling via Cycle-consistencycited by 7
  5. 2022StreamYOLO: Real-time Object Detection for Streaming Perceptioncited by 7
  6. 2021Review of Video Predictive Understanding: Early Action Recognition and Future Action Predictioncited by 5
  7. 2025VFMF: World Modeling by Forecasting Vision Foundation Model Featurescited by 5
  8. 2021Multi-Camera Trajectory Forecasting with Trajectory Tensorscited by 4
  9. 2022Forecasting Future Instance Segmentation with Learned Optical Flow and Warpingcited by 4
  10. 2025Frozen Forecasting: A Unified Evaluationcited by 4
  11. 2021Dense Semantic Forecasting in Video by Joint Regression of Features and Feature Motioncited by 3
  12. 2021Hierarchical Video Generation for Complex Datacited by 3
  13. 2022Joint Forecasting of Panoptic Segmentations with Difference Attentioncited by 3
  14. 2022Patch-based Object-centric Transformers for Efficient Video Generationcited by 3
  15. 2022How Far Can I Go ? : A Self-Supervised Approach for Deterministic Video Depth Forecastingcited by 3
  16. 2023FLODCAST: Flow and Depth Forecasting via Multimodal Recurrent Architecturescited by 2
  17. 2024Panoptic-Depth Forecastingcited by 1
  18. 2022Borrowing from yourself: Faster future video segmentation with partial channel updatecited by 0
  19. 2022Future Object Detection with Spatiotemporal Transformerscited by 0
  20. 2022Cascaded Video Generation for Videos In-the-Wildcited by 0
  21. 2022Learning a Fast 3D Spectral Approach to Object Segmentation and Tracking over Space and Timecited by 0
  22. 2023Perceive, Excavate and Purify: A Novel Object Mining Framework for Instance Segmentationcited by 0
  23. 20233D Pose Nowcasting: Forecast the Future to Improve the Presentcited by 0
  24. 2024Frame forecasting in cine MRI using the PCA respiratory motion model: comparing recurrent neural networks trained online and transformerscited by 0
  25. 2024Parallel Neural Computing for Scene Understanding from LiDAR Perception in Autonomous Racingcited by 0