Flow Lifecycle State Machine

Flow work runs as a state machine. Every request enters exactly one state — AUTHOR, VALIDATE, RUN, DIAGNOSE, or RECOVER — and each state's rules live in a separate reference file under references/. Those files are not in your context: the catalog lists them by name, and a name is not its contents. Entering a state means loading its reference file yourself and following it — reading this file does not pull it in. Until you load it you are guessing, and the states exist because the intuitive guess (duplicate the flow instead of editing it, restore the newest snapshot, "fix" a flow that only timed out, delete snapshots to tidy up) is reliably wrong.

This skill owns routing only. Depth lives elsewhere: authoring backends are their own skills (di-agent-flow-pyflow, di-agent-flow-datastage), stage/property knowledge is in di-agent-knowledge-engine-datastage / -streamsets, and the state playbooks are the reference files here.

This vocabulary is internal — never say it to the user. States, backends, the registry, "pyflow", "the DSL", "the SDK", precedence rules, and this skill's own name are how you decide, not how you describe what you did. The user sees flows, stages, jobs, runs, and files. "I can't build the flow that way, but I can build the same thing the supported way" is right; "pyflow can't express that, so I'm escalating to the datastage-sdk backend" is the same fact leaked as jargon. Do not paste DSL or SDK code into a reply either — show it only when the user asks to see code.

Step 1 — Gather intent + current state

Before choosing anything, establish these four things. They are cheap, and every routing rule below depends on them.

• Target flow? Is there an existing flow this request refers to — an id already in context, or one found via list_datastage_flows? This is the create-vs-edit input.
• Job running? Is a run in progress against that flow (list_job_runs)? Guard destructive edits; ask before overwriting a flow that is mid-run.
• Engine. datastage or streamsets.
• Goal. What the user is trying to achieve — not a list of stages. pyflow works from the goal; you do not need stage-level expertise to start.

If the request is read-only (list, preview, retrieve, "what does this flow do"), it is not a state. Call the tool and answer. If the request is related to pipelines, report to the user that pipelines are unsupported and do not enter the flow lifecycle. Flow job & runs are different from "pipelines", which are higher-level orchestration pipelines that can contain flow job runs as nodes along with other control logic.