Optimize Environment Performance

Goal

Find and fix synchronous bottlenecks in verifiers environment code so that rollouts scale efficiently under concurrency. The verifiers runtime runs all rollouts on a single async event loop — any sync operation blocks every concurrent rollout.

Audit Workflow

1. Identify Async Entry Points

Locate all async methods in the environment (typically setup_state, env_response, score, cleanup, and any tool functions). These are the hot paths where sync operations cause the most damage.

2. Scan for Sync Offenders

Search for these patterns inside async methods, ordered by typical severity:

Critical — blocks network I/O:

• time.sleep() → replace with await asyncio.sleep()
• Sync HTTP clients (requests, httpx.Client, urllib) → replace with httpx.AsyncClient or equivalent
• Sync LLM clients (OpenAI(), litellm.completion()) → replace with AsyncOpenAI() or use self.get_model_response()