Benchmarks

Research

Benchmarking suite to evaluate 🤖 robotics computing performance. Vendor-neutral. ⚪Grey-box and ⚫Black-box approaches.

Research

MapEval: Towards Unified, Robust and Efficient SLAM Map Evaluation Framework.

Research

Open datasets for training and benchmarking AI and robotic systems in agriculture and off-road environments

Research

Evolution Gym benchmarks joint optimization of soft-robot design and control.

Research

Faultline studies LIBERO policy robustness using perturbation and ablation workflows.

Research

GeoPolicy-Bench compares perception and policy designs on Panda manipulation tasks.

Research

A simulation environment and benchmark for human-to-robot object handovers

Research

Open source evals for physical AI. Run any LLM/VLA on any arm/humanoid against any real/sim benchmark.

Research

An All-in-one robot manipulation learning suite for policy models training and evaluation on various datasets and benchmarks.

Research

Isaac Lab - Arena is a robotics simulation framework that enhances NVIDIA Isaac Lab by providing a composable, scalable system for creating diverse simulation environments and evaluating robot…

Research

LIBERO Camera Ablation studies how camera-input changes affect robot-policy behavior.

Research

Generate and benchmark datasets for robot manipulation problems

Research

OceanGym: A Benchmark Environment for Underwater Embodied Agents

Research

Open-source robotics simulator for coding agents: HTTP/JSON + MCP control, Newton physics, wgpu rendering, ROS 2, and reproducible benchmarks.

Research

ParcelStow evaluates imitation-learned robot-policy robustness under execution-timing variation in Isaac Lab.

Research

Pre-Execution Return studies robot initial-state robustness around frozen VLA policies.

Research

A research benchmark for robot manipulation policy evaluation.

Research

Repository associated with paper titled "RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis"

Research

A simulation benchmarking platform for evaluating generalist robot policies

Research

A software framework integrating various imitation learning methods and benchmark environments for robotic manipulation

Research

RoboPianist Cost-Aware Intervention provides a benchmark for teacher interventions in RoboPianist.

Research

RoboSense: Large-scale Dataset and Benchmark for Egocentric Robot Perception and Navigation in Crowded and Unstructured Environments

Research

100+ robotics algorithms in Rust: planning, localization, SLAM, control, and benchmarks.

Research

SLAM performance evaluation framework