A robot policy can look reliable in a standard simulation and still stumble when the camera view changes. If your setup needs a different camera position, look past the leaderboard average and check what happens under changes to the camera. LIBERO-Plus shows why.

Concept illustration. No real-robot footage.
A score is only as useful as its test conditions
LIBERO is a simulation benchmark for studying how robot policies transfer knowledge across tasks. Its suites vary spatial relationships, objects and goals. Two scores are directly comparable only when the tasks, starting conditions and scoring rules match.
Table 1 of LIBERO-Plus v2 asks what happens to the same policy when one category of input conditions changes. In the authors’ simulation, OpenVLA-OFT scored 97.1% without perturbation and 59.7% on the camera perturbation tests, a gap of 37.4 percentage points. The tests cover changes in camera distance, position and orientation. They do not represent one camera rotation or physical-robot results.

Chart redrawn from LIBERO-Plus v2, Table 1. Not a BotsTest measurement.
Change one condition, not the whole test
Before moving to hardware, run two sets of simulation trials. First, keep the camera fixed and lock the task, checkpoint, initial states, success rule and timeout. Then repeat with only the predefined camera conditions changed, recording every trial. If the baseline is unstable, investigate the evaluation setup before blaming the camera.
When comparing policies, use the same tasks and initial states. The LeRobot evaluation guide recommends using the same random seed and fixed initial states. It also notes that soft and hard resets do not produce bit-for-bit identical initial observations. Don’t use different reset methods for the two policies.

BotsTest’s suggested simulation test procedure.
Keep the failures, not just the good clips
Log successful trials / total trials and where the failure happened. A run that never locates the target differs from one that grasps it but never places it, or one that times out. These are observable outcomes, not evidence that the robot understood or refused an instruction. The distinction tells you whether to check the input, the grasp or the controls that come afterward.

Example evaluation log. It contains no experimental data.
If the baseline is stable but trials fail frequently under your intended camera view, retest under that view before committing hardware time. Passing this simulation check does not establish deployment readiness. It says only that the tested changes did not expose an obvious weakness. BotsTest reviewed published results and did not run these experiments.
Sources
- BotsTest: LIBERO resource entry
- LIBERO project and task suites
- LIBERO-Plus v2, Table 1 and Appendix A.4
- LeRobot evaluation guide
The results chart redraws paper data. The other figures are original BotsTest illustrations.
Turn the shortlist into a decision
Write down the task, operating conditions, success criteria and total ownership costs. Ask shortlisted suppliers to demonstrate your representative task, including failure recovery. Use the same acceptance test for each candidate.
Explore the full robotics directory →