mlsanityData to 7 Oct 2026

Surrogate Assisted Generation of Human-Robot Interaction Scenarios

arXiv 2304.13787 ↗First posted 26 Apr 2023cs.RO cs.HC cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

As human-robot interaction (HRI) systems advance, so does the difficulty of evaluating and understanding the strengths and limitations of these systems in different environments and with different users. To this end, previous methods have algorithmically generated diverse scenarios that reveal system failures in a shared control teleoperation task. However, these methods require directly evaluating generated scenarios by simulating robot policies and human actions. The computational cost of these evaluations limits their applicability in more complex domains. Thus, we propose augmenting scenario generation systems with surrogate models that predict both human and robot behaviors. In the shared control teleoperation domain and a more complex shared workspace collaboration task, we show that surrogate assisted scenario generation efficiently synthesizes diverse datasets of challenging scenarios. We demonstrate that these failures are reproducible in real-world interactions.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 51 of its 76 references that are in the map. First posted on arXiv 26 Apr 2023.

Builds on

51 of 76 references.

References · 51

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  4. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  5. 2015Illuminating search spaces by mapping elitescited by 406
  6. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  7. 2016The CMA Evolution Strategy: A Tutorialcited by 364
  8. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  9. 2020Prioritized Level Replaycited by 190
  10. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  11. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  12. 2019Active Domain Randomizationcited by 161
38 more
  1. 2021Replay-Guided Adversarial Environment Designcited by 108
  2. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 103
  3. 2018Scenic: A Language for Scenario Specification and Scene Generationcited by 79
  4. 2019Covariance Matrix Adaptation for the Rapid Illumination of Behavior Spacecited by 78
  5. 2021Differentiable Quality Diversitycited by 71
  6. 2016Using Centroidal Voronoi Tessellations to Scale Up the Multi-dimensional Archive of Phenotypic Elites Algorithmcited by 65
  7. 2019Increasing Generality in Machine Learning through Procedural Content Generationcited by 58
  8. 2019Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptorscited by 57
  9. 2020Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimizationcited by 53
  10. 2020Illuminating Mario Scenes in the Latent Space of a Generative Adversarial Networkcited by 51
  11. 2015Shared Autonomy via Hindsight Optimizationcited by 46
  12. 2019Procedural Content Generation through Quality Diversitycited by 45
  13. 2019Assistive Gym: A Physics Simulation Framework for Assistive Roboticscited by 44
  14. 2022Deep Surrogate Assisted Generation of Environmentscited by 40
  15. 2017Shared Autonomy via Hindsight Optimization for Teleoperation and Teamingcited by 39
  16. 2018Data-Efficient Design Exploration through Surrogate-Assisted Illuminationcited by 38
  17. 2021On the Importance of Environments in Human-Robot Coordinationcited by 37
  18. 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 37
  19. 2021Illuminating Diverse Neural Cellular Automata for Level Generationcited by 35
  20. 2020Shared Autonomy with Learned Latent Actionscited by 34
  21. 2018Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planningcited by 32
  22. 2021Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recoverycited by 32
  23. 2023pyribs: A Bare-Bones Python Library for Quality Diversity Optimizationcited by 29
  24. 2021Generating and Blending Game Levels via Quality-Diversity in the Latent Space of a Variational Autoencodercited by 27
  25. 2018Talakat: Bullet Hell Generation through Constrained Map-Elitescited by 26
  26. 2021Deep Surrogate Assisted MAP-Elites for Automated Hearthstone Deckbuildingcited by 26
  27. 2022Covariance Matrix Adaptation MAP-Annealingcited by 24
  28. 2020A Quality Diversity Approach to Automatically Generating Human-Robot Interaction Scenarios in Shared Autonomycited by 20
  29. 2020CPPN2GAN: Combining Compositional Pattern Producing Networks and GANs for Large-scale Pattern Generationcited by 19
  30. 2020Video Game Level Repair via Mixed Integer Linear Programmingcited by 18
  31. 2020Co-generation of game levels and game-playing agentscited by 14
  32. 2021Illuminating the Space of Beatable Lode Runner Levels Produced By Various Generative Adversarial Networkscited by 14
  33. 2022RCareWorld: A Human-centric Simulation World for Caregiving Robotscited by 14
  34. 2021Designing Air Flow with Surrogate-assisted Phenotypic Nichingcited by 11
  35. 2020QED: using Quality-Environment-Diversity to evolve resilient robot swarmscited by 8
  36. 2020RoCUS: Robot Controller Understanding via Samplingcited by 6
  37. 2022Transfer Dynamics in Emergent Evolutionary Curriculacited by 6
  38. 2021Discovering Generalizable Skills via Automated Generation of Diverse Taskscited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 1; 2026: 2.

8 papers.

  1. 2023Quality-Diversity through AI Feedbackcited by 40
  2. 2024CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectoriescited by 11
  3. 2025Efficient Evaluation of Multi-Task Robot Policies With Active Experiment Selectioncited by 7
  4. 2024Algorithmic Scenario Generation as Quality Diversity Optimizationcited by 1
  5. 2024QD-MAPPER: A Quality Diversity Framework to Automatically Evaluate Multi-Agent Path Finding Algorithms in Diverse Mapscited by 1
  6. 2026Discount Model Search for Quality Diversity Optimization in High-Dimensional Measure Spacescited by 1
  7. 2026Red-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policiescited by 1
  8. 2024SocRATES: Towards Automated Scenario-based Testing of Social Navigation Algorithmscited by 0