mlsanityData to 7 Oct 2026

Modality Selection and Skill Segmentation via Cross-Modality Attention

arXiv 2504.14573 ↗First posted 20 Apr 2025cs.RO cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Incorporating additional sensory modalities such as tactile and audio into foundational robotic models poses significant challenges due to the curse of dimensionality. This work addresses this issue through modality selection. We propose a cross-modality attention (CMA) mechanism to identify and selectively utilize the modalities that are most informative for action generation at each timestep. Furthermore, we extend the application of CMA to segment primitive skills from expert demonstrations and leverage this segmentation to train a hierarchical policy capable of solving long-horizon, contact-rich manipulation tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 20 Apr 2025.

Builds on

27 of 33 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  9. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  10. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  11. 2020Integrated Task and Motion Planningcited by 281
  12. 2024Diffusion Policy Policy Optimizationcited by 255
15 more
  1. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 229
  2. 2018Residual Policy Learningcited by 146
  3. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  4. 2017Behavior Trees in Robotics and AI: An Introductioncited by 105
  5. 2022ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfercited by 73
  6. 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representationscited by 67
  7. 2022See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulationcited by 61
  8. 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 59
  9. 2023The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objectscited by 40
  10. 2025Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Groundingcited by 40
  11. 2021Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordancescited by 38
  12. 2024JUICER: Data-Efficient Imitation Learning for Robotic Assemblycited by 36
  13. 2024Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulationcited by 19
  14. 2024DexSkills: Skill Segmentation Using Haptic Data for Learning Autonomous Long-Horizon Robotic Manipulation Taskscited by 9
  15. 2024Generalize by Touching: Tactile Ensemble Skill Transfer for Robotic Furniture Assemblycited by 6

Built on by

Not cited yet.