streamling advanced plugin kinds — Agent Skill

Beyond sources, transforms, and sinks, a plugin crate can register three more component kinds — preprocessors, UDFs, and side outputs — and can mix all six in one cdylib. This skill also covers the low-level manual FFI escape hatch for when the registration macros don't fit.

Prerequisite: streamling-plugin-basics.

Preprocessors — rewrite topology YAML before parsing

A preprocessor transforms the raw topology config string (the pipeline YAML) before streamling parses it. Use it to inject default options, expand shorthand dataset names into concrete source configs, or enforce per-source conventions — without touching streamling's core config parser.

use async_trait::async_trait;
use std::collections::HashMap;
use streamling_plugin::api::PluginError;
use streamling_plugin::{PluginInitializationError, PreprocessorPlugin};

pub struct DefaultsPreprocessor {
    default_region: String,
}