# Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction

arXiv: https://arxiv.org/abs/2204.08687 · HTML: https://mlsanity.com/paper/2204.08687/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation](https://mlsanity.com/paper/1811.02790/) (2018)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [Learning from Dialogue after Deployment: Feed Yourself, Chatbot!](https://mlsanity.com/paper/1901.05415/) (2019)
- [Efficient Continual Learning with Modular Networks and Task-Driven Priors](https://mlsanity.com/paper/2012.12631/) (2020)
- [Executing Instructions in Situated Collaborative Interactions](https://mlsanity.com/paper/1910.03655/) (2019)
- [Machine Teaching: A New Paradigm for Building Machine Learning Systems](https://mlsanity.com/paper/1707.06742/) (2017)
- [The MineRL BASALT Competition on Learning from Human Feedback](https://mlsanity.com/paper/2107.01969/) (2021)
- [CoSTAR: Instructing Collaborative Robots with Behavior Trees and Vision](https://mlsanity.com/paper/1611.06145/) (2016)
- [Naturalizing a Programming Language via Interactive Learning](https://mlsanity.com/paper/1704.06956/) (2017)
- [Megaverse: Simulating Embodied Agents at One Million Experiences per Second](https://mlsanity.com/paper/2107.08170/) (2021)
- [Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior](https://mlsanity.com/paper/2108.04812/) (2021)
- [RoboBrain: Large-Scale Knowledge Engine for Robots](https://mlsanity.com/paper/1412.0691/) (2014)
- [Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models](https://mlsanity.com/paper/1911.03782/) (2019)
- [droidlet: modular, heterogenous, multi-modal agents](https://mlsanity.com/paper/2101.10384/) (2021)

## Built on by (most cited first, 1 of 1)

- [IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents](https://mlsanity.com/paper/2407.08898/) (2024)

## Abstract (arXiv)

In this work we give a case study of an embodied machine-learning (ML) powered agent that improves itself via interactions with crowd-workers. The agent consists of a set of modules, some of which are learned, and others heuristic. While the agent is not "end-to-end" in the ML sense, end-to-end interaction is a vital part of the agent's learning mechanism. We describe how the design of the agent works together with the design of multiple annotation interfaces to allow crowd-workers to assign credit to module errors from end-to-end interactions, and to label data for individual modules. Over multiple automated human-agent interaction, credit assignment, data annotation, and model re-training and re-deployment, rounds we demonstrate agent improvement.