generate-python-runnability-test

Installation
SKILL.md

Generate Python Runnability Test

Use this skill to create the tests/test_runnability.py file that every Python recipe under core/python/, contrib/python/, or skills/<vertical>/<solution>/ must ship (see python-validate-recipe.yml Check 4). The generated test is deliberately minimal — it just verifies the agent module imports and defines the expected globals. Business-logic testing lives elsewhere.


What This Skill Does

Runs scripts/generate_runnability_test.py against a recipe directory. Steps:

  1. Locate agent.py. Walks the recipe safely (excludes .venv, venv, env, build, dist, __pycache__, node_modules, tests, *.egg-info, and dot-directories) and picks the shallowest agent.py match. If none is found, errors out and suggests --agent-file. Only one file is generated per invocation.

  2. Parse agent.py AND every ancestor package __init__.py with ast to detect:

    • Top-level assignments (agent.py only, per convention) — is root_agent = ... present? Is app = ... present?
    • Import-time calls (agent.py + every ancestor __init__.py) — does vertexai.init(...) fire at module load? Does google.auth.default()? Every ancestor __init__.py is checked because Python runs each of them in order when the test does import a.b.agent (a/__init__.py, then a/b/__init__.py, then the module), so a side effect in any of them matters as much as one in agent.py. Historical bug closed by this: cross-session-memory has _, project_id = google.auth.default() in __init__.py; before, the scanner missed it and the generated test crashed in CI without ADC. Detection uses ast.walk (any depth), so it is intentionally broad — a call nested in a function body still flags the recipe; the resulting patch is a harmless no-op if it never fires at import time, whereas a missed import-time call would crash the generated test.
    • Env-var access (agent.py + every ancestor __init__.py) — does the code read GOOGLE_CLOUD_PROJECT via os.getenv / os.environ.get / os.environ["…"]? Only reads count: an os.environ["…"] = value write means the recipe sets its own value and doesn't depend on the test providing one, so it's ignored.
  3. Scan all source .py files in the recipe directory tree (same safe walker) for INTEGRATION_TEST env-var reads. This is a per-package convention: agent.py often calls a helper (e.g. retrievers.create_search_tool) at module load, and THAT helper — which may live anywhere in the recipe tree, not necessarily beside agent.py — is what reads INTEGRATION_TEST. Restricting the scan to agent.py would miss it.

Installs
3
GitHub Stars
10.4K
First Seen
Aug 15, 2026
generate-python-runnability-test — google/adk-recipes