generate-python-runnability-test
Generate Python Runnability Test
Use this skill to create the tests/test_runnability.py file that every Python recipe under core/python/, contrib/python/, or skills/<vertical>/<solution>/ must ship (see python-validate-recipe.yml Check 4). The generated test is deliberately minimal — it just verifies the agent module imports and defines the expected globals. Business-logic testing lives elsewhere.
What This Skill Does
Runs scripts/generate_runnability_test.py against a recipe directory. Steps:
-
Locate
agent.py. Walks the recipe safely (excludes.venv,venv,env,build,dist,__pycache__,node_modules,tests,*.egg-info, and dot-directories) and picks the shallowestagent.pymatch. If none is found, errors out and suggests--agent-file. Only one file is generated per invocation. -
Parse
agent.pyAND every ancestor package__init__.pywithastto detect:- Top-level assignments (agent.py only, per convention) — is
root_agent = ...present? Isapp = ...present? - Import-time calls (agent.py + every ancestor
__init__.py) — doesvertexai.init(...)fire at module load? Doesgoogle.auth.default()? Every ancestor__init__.pyis checked because Python runs each of them in order when the test doesimport a.b.agent(a/__init__.py, thena/b/__init__.py, then the module), so a side effect in any of them matters as much as one inagent.py. Historical bug closed by this: cross-session-memory has_, project_id = google.auth.default()in__init__.py; before, the scanner missed it and the generated test crashed in CI without ADC. Detection usesast.walk(any depth), so it is intentionally broad — a call nested in a function body still flags the recipe; the resulting patch is a harmless no-op if it never fires at import time, whereas a missed import-time call would crash the generated test. - Env-var access (agent.py + every ancestor
__init__.py) — does the code readGOOGLE_CLOUD_PROJECTviaos.getenv/os.environ.get/os.environ["…"]? Only reads count: anos.environ["…"] = valuewrite means the recipe sets its own value and doesn't depend on the test providing one, so it's ignored.
- Top-level assignments (agent.py only, per convention) — is
-
Scan all source
.pyfiles in the recipe directory tree (same safe walker) forINTEGRATION_TESTenv-var reads. This is a per-package convention:agent.pyoften calls a helper (e.g.retrievers.create_search_tool) at module load, and THAT helper — which may live anywhere in the recipe tree, not necessarily besideagent.py— is what readsINTEGRATION_TEST. Restricting the scan toagent.pywould miss it.