Benchmarking a Studio Policy

Benchmarking evaluates a trained policy by rolling it out in a gym and scoring success. Benchmark classes live in library/src/physicalai/benchmark/gyms/benchmark.py (Benchmark, PushTBenchmark, LiberoBenchmark); results types in benchmark/gyms/results.py (BenchmarkResults, TaskResult); rollout logic in library/src/physicalai/eval/rollout.py (evaluate_policy). The library supports both direct Python API use and the physicalai benchmark CLI wrapper (library/src/physicalai/cli/benchmark.py).

Python API invocation

Use this path for notebooks, tests, custom scripts, or direct library integrations.

from physicalai.benchmark.gyms import PushTBenchmark
from physicalai.policies import ACT

policy = ACT.load_from_checkpoint("experiments/act/version_0/checkpoints/last.ckpt")
benchmark = PushTBenchmark(num_episodes=1)
results = benchmark.evaluate(policy)
print(results.summary())
results.to_json("results/benchmark/results.json")
results.to_csv("results/benchmark/results.csv")