# SIERRA: A Modular Framework for Research Automation and Reproducibility

arXiv: https://arxiv.org/abs/2208.07805 · HTML: https://mlsanity.com/paper/2208.07805/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 25 references that are in the map.
- First posted on arXiv 16 Aug 2022.

## Builds on (most cited first, 6 of 6)

- [The Robotarium: A remotely accessible swarm robotics research testbed](https://mlsanity.com/paper/1609.04730/) (2016)
- [Improving Accountability in Recommender Systems Research Through Reproducibility](https://mlsanity.com/paper/2102.00482/) (2021)
- [Swarm Engineering Through Quantitative Measurement of Swarm Robotic Principles in a 10,000 Robot Swarm](https://mlsanity.com/paper/1907.03880/) (2019)
- [Webots.HPC: A Parallel Robotics Simulation Pipeline for Autonomous Vehicles on High Performance Computing](https://mlsanity.com/paper/2108.00485/) (2021)
- [Characterizing The Limits of Linear Modeling of Non-Linear Swarm Behaviors](https://mlsanity.com/paper/2110.12307/) (2021)
- [SIERRA: A Modular Framework for Research Automation](https://mlsanity.com/paper/2203.04748/) (2022)

## Built on by (most cited first, 1 of 1)

- [Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease](https://mlsanity.com/paper/2505.23852/) (2025)

## Abstract (arXiv)

Modern intelligent systems researchers form hypotheses about system behavior and then run experiments using one or more independent variables to test their hypotheses. We present SIERRA, a novel framework structured around that idea for accelerating research development and improving reproducibility of results. SIERRA accelerates research by automating the process of generating executable experiments from queries over independent variables(s), executing experiments, and processing the results to generate deliverables such as graphs and videos. It shifts the paradigm for testing hypotheses from procedural ("Do these steps to answer the query") to declarative ("Here is the query to test--GO!"), reducing the burden on researchers. It employs a modular architecture enabling easy customization and extension for the needs of individual researchers, thereby eliminating manual configuration and processing via throw-away scripts. SIERRA improves reproducibility of research by providing automation independent of the execution environment (HPC hardware, real robots, etc.) and targeted platform (arbitrary simulator or real robots). This enables exact experiment replication, up to the limit of the execution environment and platform, as well as making it easy for researchers to test hypotheses in different computational environments.