Benchmarking a Studio Policy

Benchmarking evaluates a trained policy by rolling it out in a gym and scoring success. PushTBenchmark and LiberoBenchmark are available through the Python API, alongside BenchmarkResults and TaskResult. Use an installed physicalai-train package or the Studio benchmark examples for CLI configs.

Python API invocation

Use this path for notebooks, tests, custom scripts, or direct library integrations.

from physicalai.benchmark.gyms import PushTBenchmark
from physicalai.policies import ACT

policy = ACT.load_from_checkpoint("experiments/act/version_0/checkpoints/last.ckpt")
benchmark = PushTBenchmark(num_episodes=1)
results = benchmark.evaluate(policy)
print(results.summary())
results.to_json("results/benchmark/results.json")
results.to_csv("results/benchmark/results.csv")