add-datalake-consumer
Installation
SKILL.md
Add Data Lake Consumer
Event consumer that subscribes to RabbitMQ and writes to Azure Data Lake (Parquet). Pattern: logging’s DataLakeCollector for risk.evaluated (BI_SALES_RISK_IMPLEMENTATION_PLAN §3.5, §9.1). BI Sales Risk: Paths and Parquet columns MUST match documentation/requirements/BI_SALES_RISK_DATA_LAKE_LAYOUT.md (§2.1 risk.evaluated, §2.2 ml_outcomes, §4 config).
1. Consumer
Path: src/events/consumers/[Name]DataLakeCollector.ts
EventConsumerwithqueue,exchange: coder_events,bindings: e.g.['risk.evaluated','ml.prediction.completed','opportunity.updated','forecast.generated'].- Handler: map event to row. For risk.evaluated use columns in Data Lake Layout §2.1. Build path:
{path_prefix}/year={YYYY}/month={MM}/day={DD}/...(Layout §1). - Write via
@azure/storage-blob(BlockBlob) or@azure/storage-blob+parquetjs(or Arrow) for Parquet. Buffer/batch by time or count if needed. - Config:
data_lake.connection_string,data_lake.container,data_lake.path_prefix(e.g./risk_evaluations).