SDK / run.py

from ageval_sdk import RunContext, RunTerminal

async def run(ctx: RunContext) -> RunTerminal:
    async with ctx.agent.session("solver", max_turns=1) as session:
        reply = await session.invoke(ctx.params["instruction"])
    ctx.publish_json("reply", {"ok": bool(reply.get("ok")), "text": reply.get("text") or ""})
    if not reply.get("ok"):
        return RunTerminal.failed(str(reply.get("error") or "invoke_failed"))
    return RunTerminal.completed("ok")

Default evaluator.py is a script: read artifacts + gold, return {status, score, metrics}. Optional extra key checks (use evaluation_check(...)) — parent writes evaluation/checks.json; bind ignores it. Optional LLM-as-judge: after gold is uploaded, Agent.session(<role>).invoke on a role declared in the same profiles.yaml. Parent injects inputs["agent"] when the socket is projected (environment: local). Still return {status, score, metrics}. invoke kwargs must not override profile_id / executor.