streamling transform plugin — Agent Skill

A transform sits in the middle of a pipeline: it receives an upstream RecordBatch, returns a (possibly reshaped) RecordBatch, and propagates checkpoints. Transforms are stateless-by-default row pipelines — use one when the built-in sql transform can't express the logic (custom Rust, per-row external calls, non-SQL projections).

Prerequisite: streamling-plugin-basics (crate setup, registration, lifecycle, errors).

When NOT to write a transform: if the work is column projection/renaming, a UNION ALL, or a JOIN, the built-in type: sql transform (DataFusion) is simpler and faster — see sql-transform. Reach for a plugin transform only when you need imperative Rust or side effects the SQL engine can't do.

The trait