pyspark-structured-streaming
Pass
Audited by Gen Agent Trust Hub on Jul 31, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONDATA_EXFILTRATION
Full Analysis
- [EXTERNAL_DOWNLOADS]: The skill configures Spark to fetch official Kafka and Delta Lake connectors from their respective project repositories on Maven Central.
- [COMMAND_EXECUTION]: The skill uses standard Spark SQL expressions and the foreachBatch API to process data micro-batches and manage storage operations.
- [DATA_EXFILTRATION]: The skill describes data flows between Kafka brokers and cloud storage buckets (S3), which are standard destinations for the described data engineering tasks.
- [SAFE]: Ingested data is processed using predefined schemas and watermarks, which provides a layer of validation and prevents resource exhaustion from unbounded state growth.
Audit Metadata