mlsanityData to 14 Oct 2026

Tactile-Filter: Interactive Tactile Perception for Part Mating

arXiv 2303.06034 ↗First posted 10 Mar 2023cs.RO cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. Our tactile sensing provides us with a lot of information regarding contact formations as well as geometric information about objects during any interaction. With this motivation, vision-based tactile sensors are being widely used for various robotic perception and control tasks. In this paper, we present a method for interactive perception using vision-based tactile sensors for a part mating task, where a robot can use tactile sensors and a feedback mechanism using a particle filter to incrementally improve its estimate of objects (pegs and holes) that fit together. To do this, we first train a deep neural network that makes use of tactile images to predict the probabilistic correspondence between arbitrarily shaped objects that fit together. The trained model is used to design a particle filter which is used twofold. First, given one partial (or non-unique) observation of the hole, it incrementally improves the estimate of the correct peg by sampling more tactile observations. Second, it selects the next action for the robot to sample the next touch (and thus image) which results in maximum uncertainty reduction to minimize the number of interactions during the perception task. We evaluate our method on several part-mating tasks with novel objects using a robot equipped with a vision-based tactile sensor. We also show the efficiency of the proposed action selection method against a naive method. See supplementary video at https://www.youtube.com/watch?v=jMVBg_e3gLw .

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

33 of 43 references.

References · 33

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  5. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 182
  6. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 165
  7. 2018Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Taskscited by 154
  8. 2018More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touchcited by 102
  9. 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 90
  10. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 80
  11. 2016Interactive Perception: Leveraging Action in Perception and Perception in Actioncited by 78
  12. 2020TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensorscited by 63
21 more
  1. 2021An End-to-End Differentiable Framework for Contact-Aware Robot Designcited by 54
  2. 2021Tactile-RL for Insertion: Generalization to Objects of Unknown Geometrycited by 48
  3. 2019Cable Manipulation with a Tactile-Reactive Grippercited by 46
  4. 2021GelSlim3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Fingercited by 41
  5. 2022MidasTouch: Monte-Carlo inference over distributions across sliding touchcited by 33
  6. 2017Active Clothing Material Perception using Tactile Sensing and Deep Learningcited by 30
  7. 2020Tactile Dexterity: Manipulation Primitives with Tactile Feedbackcited by 23
  8. 2021Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translationcited by 19
  9. 2019Object Finding in Cluttered Scenes Using Interactive Perceptioncited by 18
  10. 2018Maintaining Grasps within Slipping Bound by Monitoring Incipient Slipcited by 16
  11. 2022Tac2Pose: Tactile Object Pose Estimation from the First Touchcited by 15
  12. 2021Extrinsic Contact Sensing with Relative-Motion Tracking from Distributed Tactile Measurementscited by 13
  13. 2019Tactile-Based Insertion for Dense Box-Packingcited by 12
  14. 2017Localizing the Object Contact through Matching Tactile Features with Visual Mapcited by 11
  15. 2023Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensorscited by 10
  16. 2021Interpreting Contact Interactions to Overcome Failure in Robot Assembly Taskscited by 9
  17. 2022Tactile Pose Estimation and Policy Learning for Unknown Object Manipulationcited by 8
  18. 2021Imitation and Supervised Learning of Compliance for Robotic Assemblycited by 6
  19. 2022Using Collocated Vision and Tactile Sensors for Visual Servoing and Localizationcited by 6
  20. 2023Simultaneous Tactile Estimation and Control of Extrinsic Contactcited by 6
  21. 2022H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactionscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 2.

4 papers.

  1. 2023Robust Pivoting Manipulation using Contact Implicit Bilevel Optimizationcited by 3
  2. 2023Tactile Estimation of Extrinsic Contact Patch for Stable Placementcited by 1
  3. 2025Zero-Shot Peg Insertion: Identifying Mating Holes and Estimating SE(2) Poses with Vision-Language Modelscited by 0
  4. 2025ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generationcited by 0