mlsanityData to 7 Oct 2026

Shift-Memory Network for Temporal Scene Segmentation

arXiv 2202.08399 ↗First posted 17 Feb 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Semantic segmentation has achieved great accuracy in understanding spatial layout. For real-time tasks based on dynamic scenes, we extend semantic segmentation in temporal domain to enhance the spatial accuracy with motion. We utilize a shift-mode network over streaming input to ensure zero-latency output. For the data overlap under shifting network, this paper identifies repeated computation in fixed periods across network layers. To avoid this redundancy, we derive a Shift-Memory Network (SMN) from encoding-decoding baseline to reuse the network values without accuracy loss. Trained in patch-mode, the SMN extracts the network parameters for SMN to perform inference promptly in compact memory. We segment dynamic scenes from 1D scanning input and 2D video. The experiments of SMN achieve equivalent accuracy as shift-mode but in faster inference speeds and much smaller memory. This will facilitate semantic segmentation in real-time application on edge devices.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 62 references that are in the map. First posted on arXiv 17 Feb 2022.

Builds on

41 of 62 references.

References · 41

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  7. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  8. 2016Pyramid Scene Parsing Networkcited by 2,472
  9. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  10. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  11. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  12. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
29 more
  1. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  2. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  4. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  5. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  6. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  7. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  8. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  9. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 483
  10. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  11. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  12. 2021Rethinking BiSeNet For Real-time Semantic Segmentationcited by 124
  13. 2018ECO: Efficient Convolutional Network for Online Video Understandingcited by 115
  14. 2019Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Drivingcited by 105
  15. 2017Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutionscited by 93
  16. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  17. 2016Semantic Video Segmentation by Gated Recurrent Flow Propagationcited by 76
  18. 2019DeeperLab: Single-Shot Image Parsercited by 63
  19. 2017Tube Convolutional Neural Network (T-CNN) for Action Detection in Videoscited by 50
  20. 2016Video Propagation Networkscited by 45
  21. 2015Human Action Recognition using Factorized Spatio-Temporal Convolutional Networkscited by 40
  22. 2021Progressive Semantic Segmentationcited by 35
  23. 2016Deep Learning Markov Random Field for Semantic Segmentationcited by 27
  24. 2021TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devicecited by 17
  25. 2018Future Semantic Segmentation with Convolutional LSTMcited by 14
  26. 2019Semantic Segmentation of Video Sequences with Convolutional LSTMscited by 14
  27. 2016Recurrent Fully Convolutional Networks for Video Segmentationcited by 10
  28. 2019An Efficient 3D CNN for Action/Object Segmentation in Videocited by 10
  29. 2020Highway Driving Dataset for Semantic Video Segmentationcited by 5

Built on by

Not cited yet.