Visual Forecasting as a Mid-level Representation for Avoidance
Not on the map. Why?
Abstract (arXiv)
The challenge of navigation in environments with dynamic objects continues to be a central issue in the study of autonomous agents. While predictive methods hold promise, their reliance on precise state information makes them less practical for real-world implementation. This study presents visual forecasting as an innovative alternative. By introducing intuitive visual cues, this approach projects the future trajectories of dynamic objects to improve agent perception and enable anticipatory actions. Our research explores two distinct strategies for conveying predictive information through visual forecasting: (1) sequences of bounding boxes, and (2) augmented paths. To validate the proposed visual forecasting strategies, we initiate evaluations in simulated environments using the Unity engine and then extend these evaluations to real-world scenarios to assess both practicality and effectiveness. The results confirm the viability of visual forecasting as a promising solution for navigation and obstacle avoidance in dynamic environments.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 17 Sep 2023.
Builds on
29 of 42 references.
References · 29
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
- 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
- 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
- 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
- 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
- 2018On Evaluation of Embodied Navigation Agentscited by 547
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
- 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
- 2022BoT-SORT: Robust Associations Multi-Pedestrian Trackingcited by 320
- 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
- 2023Track Anything: Segment Anything Meets Videoscited by 234
17 more
- 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
- 2020DACS: Domain Adaptation via Cross-domain Mixed Samplingcited by 226
- 2019Soft Actor-Critic for Discrete Action Settingscited by 174
- 2019What the Constant Velocity Model Can Teach Us About Pedestrian Motion Predictioncited by 80
- 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
- 2020Predictive Information Accelerates Learning in RLcited by 56
- 2021Learning to See before Learning to Act: Visual Pre-training for Manipulationcited by 51
- 2019Discrete and Continuous Action Representation for Practical RL in Video Gamescited by 39
- 2020Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigationcited by 32
- 2019Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervisioncited by 27
- 2023What Truly Matters in Trajectory Prediction for Autonomous Driving?cited by 17
- 2021Group-based Motion Prediction for Navigation in Crowded Environmentscited by 15
- 2023From Crowd Motion Prediction to Robot Navigation in Crowdscited by 13
- 2018Virtual-to-Real: Learning to Control in Visual Semantic Segmentationcited by 12
- 2021Winding Through: Crowd Navigation via Topological Invariancecited by 11
- 2019Visual Reaction: Learning to Play Catch with Your Dronecited by 6
- 2022Investigation of Factorized Optical Flows as Mid-Level Representationscited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.