# Bridging the Evaluation Gap: Standardized Benchmarks for Multi-Objective Search

arXiv: https://arxiv.org/abs/2603.24084 · HTML: https://mlsanity.com/paper/2603.24084/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 30 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 4 of 4)

- [An Extensible Benchmarking Infrastructure for Motion Planning Algorithms](https://mlsanity.com/paper/1412.6673/) (2014)
- [A Preprocessing Framework for Efficient Approximate Bi-Objective Shortest-Path Computation in the Presence of Correlated Objectives](https://mlsanity.com/paper/2505.22244/) (2025)
- [Generalizing Multi-Objective Search via Objective-Aggregation Functions](https://mlsanity.com/paper/2509.22085/) (2025)
- [Scalable Many-Objective Pathfinding Benchmark Suite](https://mlsanity.com/paper/2010.04501/) (2020)

## Abstract (arXiv)

Empirical evaluation in multi-objective search (MOS) has historically suffered from fragmentation, relying on heterogeneous problem instances with incompatible objective definitions that make cross-study comparisons difficult. This standardization gap is further exacerbated by the realization that DIMACS road networks, a historical default benchmark for the field, exhibit highly correlated objectives that fail to capture diverse Pareto-front structures. To address this, we introduce the first comprehensive, standardized benchmark suite for exact and approximate MOS. Our suite spans four structurally diverse domains: real-world road networks, structured synthetic graphs, game-based grid environments, and high-dimensional robotic motion-planning roadmaps. By providing fixed graph instances, standardized start-goal queries, reference exact Pareto-optimal solution sets, and a standardized approximate (varepsilon-dominance) evaluation protocol, this suite captures a full spectrum of objective interactions: from strongly correlated to strictly independent. Ultimately, this benchmark provides a common foundation to ensure future MOS evaluations are robust, reproducible, and structurally comprehensive.