# Learning to Push by Grasping: Using multiple tasks for effective learning

arXiv: https://arxiv.org/abs/1609.09025 · HTML: https://mlsanity.com/paper/1609.09025/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2016.

## Built on by (most cited first, 30 of 37)

- [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [Multi-Task Learning as a Bargaining Game](https://mlsanity.com/paper/2202.01017/) (2022)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking](https://mlsanity.com/paper/2309.01918/) (2023)
- [CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning](https://mlsanity.com/paper/2212.05711/) (2022)
- [RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation](https://mlsanity.com/paper/2402.15487/) (2024)
- [M2T2: Multi-Task Masked Transformer for Object-centric Pick and Place](https://mlsanity.com/paper/2311.00926/) (2023)
- [Towards Versatile Embodied Navigation](https://mlsanity.com/paper/2210.16822/) (2022)
- [Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Tasks](https://mlsanity.com/paper/2210.06601/) (2022)
- [MoE-Loco: Mixture of Experts for Multitask Locomotion](https://mlsanity.com/paper/2503.08564/) (2025)
- [Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms](https://mlsanity.com/paper/2103.03697/) (2021)
- [DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulation](https://mlsanity.com/paper/2503.16806/) (2025)
- [The Functional Correspondence Problem](https://mlsanity.com/paper/2109.01097/) (2021)
- [HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworks](https://mlsanity.com/paper/2103.09439/) (2021)
- [Semantically Controllable Augmentations for Generalizable Robot Learning](https://mlsanity.com/paper/2409.00951/) (2024)
- [Auxiliary Learning as an Asymmetric Bargaining Game](https://mlsanity.com/paper/2301.13501/) (2023)
- [Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks](https://mlsanity.com/paper/2507.23172/) (2025)
- [Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2311.01075/) (2023)
- [IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes](https://mlsanity.com/paper/2112.05298/) (2021)
- [Agent as Cerebrum, Controller as Cerebellum: Implementing an Embodied LMM-based Agent on Drones](https://mlsanity.com/paper/2311.15033/) (2023)
- [A Hybrid Approach for Learning to Shift and Grasp with Elaborate Motion Primitives](https://mlsanity.com/paper/2111.01510/) (2021)
- [Simple Emergent Action Representations from Multi-Task Policy Training](https://mlsanity.com/paper/2210.09566/) (2022)
- [State Representation Learning from Demonstration](https://mlsanity.com/paper/1910.01738/) (2019)
- [SLAW: Scaled Loss Approximate Weighting for Efficient Multi-Task Learning](https://mlsanity.com/paper/2109.08218/) (2021)
- [Learning Multi-Objective Curricula for Robotic Policy Learning](https://mlsanity.com/paper/2110.03032/) (2021)
- [Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization](https://mlsanity.com/paper/2205.12648/) (2022)
- [DynaShare: Task and Instance Conditioned Parameter Sharing for Multi-Task Learning](https://mlsanity.com/paper/2305.17305/) (2023)
- [Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning](https://mlsanity.com/paper/2408.15593/) (2024)
- [Efficient Multi-Task Reinforcement Learning via Task-Specific Action Correction](https://mlsanity.com/paper/2404.05950/) (2024)

## Abstract (arXiv)

Recently, end-to-end learning frameworks are gaining prevalence in the field of robot control. These frameworks input states/images and directly predict the torques or the action parameters. However, these approaches are often critiqued due to their huge data requirements for learning a task. The argument of the difficulty in scalability to multiple tasks is well founded, since training these tasks often require hundreds or thousands of examples. But do end-to-end approaches need to learn a unique model for every task? Intuitively, it seems that sharing across tasks should help since all tasks require some common understanding of the environment. In this paper, we attempt to take the next step in data-driven end-to-end learning frameworks: move from the realm of task-specific models to joint learning of multiple robot tasks. In an astonishing result we show that models with multi-task learning tend to perform better than task-specific models trained with same amounts of data. For example, a deep-network learned with 2.5K grasp and 2.5K push examples performs better on grasping than a network trained on 5K grasp examples.