Tactile MNIST: Benchmarking Active Tactile Perception
Not on the map. Why?
Abstract (arXiv)
Tactile perception has the potential to significantly enhance dexterous robotic manipulation by providing rich local information that can complement or substitute for other sensory modalities such as vision. However, because tactile sensing is inherently local, it is not well-suited for tasks that require broad spatial awareness or global scene understanding on its own. A human-inspired strategy to address this issue is to consider active perception techniques instead. That is, to actively guide sensors toward regions with more informative or significant features and integrate such information over time in order to understand a scene or complete a task. Both active perception and different methods for tactile sensing have received significant attention recently. Yet, despite advancements, both fields lack standardized benchmarks. To bridge this gap, we introduce the Tactile MNIST Benchmark Suite, an open-source, Gymnasium-compatible benchmark specifically designed for active tactile perception tasks, including localization, classification, and volume estimation. Our benchmark suite offers diverse simulation scenarios, from simple toy environments all the way to complex tactile perception tasks using vision-based tactile sensors. Furthermore, we also offer a comprehensive dataset comprising 13,500 synthetic 3D MNIST digit models and 153,600 real-world tactile samples collected from 600 3D printed digits. Using this dataset, we train a CycleGAN for realistic tactile simulation rendering. By providing standardized protocols and reproducible evaluation frameworks, our benchmark suite facilitates systematic progress in the fields of tactile sensing and active perception.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 39 of its 70 references that are in the map. First posted on arXiv 3 Jun 2025.
Builds on
39 of 70 references.
References · 39
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
- 2020DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulationcited by 167
- 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 160
- 2023General In-Hand Object Rotation with Vision and Touchcited by 99
- 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
- 2016Revisiting Active Perceptioncited by 94
- 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 87
- 2023Rotating without Seeing: Towards In-hand Dexterity through Touchcited by 84
27 more
- 2020HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisationcited by 83
- 2022ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfercited by 75
- 2024A Touch, Vision, and Language Dataset for Multimodal Alignmentcited by 69
- 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representationscited by 68
- 2019Connecting Touch and Vision via Cross-Modal Predictioncited by 66
- 2024Sparsh: Self-supervised touch representations for vision-based tactile sensingcited by 64
- 2020TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensorscited by 63
- 2017A Dataset for Developing and Benchmarking Active Visioncited by 50
- 2024Octopi: Object Property Reasoning with Large Tactile-Language Modelscited by 48
- 2021Taxim: An Example-based Simulation Model for GelSight Tactile Sensorscited by 43
- 2023The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objectscited by 41
- 2022Self-Supervised Visuo-Tactile Pretraining to Locate and Follow Garment Featurescited by 38
- 2022MidasTouch: Monte-Carlo inference over distributions across sliding touchcited by 33
- 2017Active Clothing Material Perception using Tactile Sensing and Deep Learningcited by 30
- 2018ViTac: Feature Sharing between Vision and Tactile Sensing for Cloth Texture Recognitioncited by 27
- 2024Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representationcited by 27
- 2017Connecting Look and Feel: Associating the visual and tactile properties of physical materialscited by 22
- 2021Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translationcited by 19
- 2022Tactile Gym 2.0: Sim-to-real Deep Reinforcement Learning for Comparing Low-cost High-Resolution Robot Touchcited by 18
- 2022TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensorscited by 12
- 2023AcTExplore: Active Tactile Exploration of Unknown Objectscited by 12
- 2023Evetac: An Event-based Optical Tactile Sensor for Robotic Manipulationcited by 12
- 2020The Robotic Vision Scene Understanding Challengecited by 9
- 2021Simultaneous Tactile Exploration and Grasp Refinement for Unknown Objectscited by 9
- 2024What Matters for Active Texture Recognition With Vision-Based Tactile Sensorscited by 7
- 2019Learning efficient haptic shape exploration with a rigid tactile sensor arraycited by 6
- 2020Introducing a new high-resolution handwritten digits data set with writer characteristicscited by 4
Built on by
Citing papers per year: 2026: 3.
3 papers.