Generate Substrait DSL

Translate natural language data requests into executable Substrait JSON via the Substrait DSL.

Inputs Required

We need the flow description and the input schema. It is important to get the input schema correct and it should be consistent with the names and types in the asset browser. Use list_project_assets to find the relevant asset(s) by names and inspect_project_asset with the asset ID(s) to retrieve the authoritative column names, types, and nullability. Do not use any other information like fewshot examples or prior knowledge to guess the schema — it must be derived from the project assets to ensure the generated Substrait plan is valid against the actual data.

If the user provides the table schemas (one or more source tables with column names and substrait types), verify them against the actual asset in the project using list_project_assets and inspect_project_asset. If there are inconsistencies, flag them to the user. The schema from the asset tools takes precedence unless the user explicitly overrides it.

If the user does not supply table schemas, derive them directly from the project assets by calling list_project_assets and inspect_project_asset.

Once you have the assets build a NamedStruct by mapping them assets to DSL types using the following mapping:

Asset Type Mapping

When constructing a NamedStruct from an inspected asset, map asset column types to DSL types as follows: